Karolina is Co-founder and Co-Executive Director at Charity Entrepreneurship. She also serves as a Fund Manager at the EA Animal Welfare Fund, and as a board member and consultant for various EA nonprofits and think tanks.
LinkedIn: https://www.linkedin.com/in/karolinasarek/
[previous comment is deleted, because I accidentally sent an unfinished one]
Thanks for the example! That makes sense and makes me wonder if part of the disagreement came from thinking about different reference classes. I agree that, in general, the research we did in our first year of operations, so 2018/2019, is well below the quality standard we expect of ourselves now, or what we expected of ourselves even in 2020. I agree it is easy to find a lot of errors (that weren’t decision-relevant) in our research from that year. That is part of the reason they are not on the website anymore.
That being said, I still broadly support our decision not to spend more time on research that year. That’s because spending more time on it would have come at a cost of significant tradeoff. At the time, there was no other organization whose research we could have relied on, and the alternative to the assessment you mention was either to not compare interventions across species (or reduce it to a simplistic metric like “the number of animals affected” metric) or to spend more time on research and run Incubation Program a year later in which case we would have lost a year of impact and might not have started the charities we did. That would have been a big loss because for example, that year we incubated Suvita whose impact and promise were recently recognized by GiveWell that, provided Suvita with $3.3M to scale up, or we incubated Fish Welfare Initiative (FWI) and Animal Advocacy Careers a decision I still consider to be a good one (FWI is an ACE Recommended Charity, and even though I agree with its co-founders that their impact could be higher, I’m glad they exist). We also couldn’t simply hire more staff and do things more in-depth because it was our first year of operation, and there was not enough funding and other resources available for, at the time, an unproven project.
I wouldn’t want to spend more time on that, especially because one of the main principles of our research is “decision-relevance,” and the “wild bug” one-pager you mention or similar ones were not relevant. If it were, we would not have settled on something of that quality, and we would have put more time into it.
For what it is worth, I think there are things we could have done better. Specifically, we could have put more effort into communicating how little weight others should put on some of that research. We did that by stating at the top (for example, as in the wild bug one-pager you link), “these reports were 1-5 hours time-limited, depending on the animal, and thus are not fully comprehensive.” and at the time, we thought it was sufficient. But we could have stressed epistemic status even more strongly and in more places so it is clear to others that we put very little weight on it. For full transparency, we also made another mistake. We didn’t recommend working on banning/reducing bait fish as an idea at the time because, from our shallow research, it looked less promising, and later, upon researching it more in-depth, we decided to recommend it. It wouldn’t have made a difference then because there were not enough potential co-founders in year 1 to start more charities, but it was a mistake, nevertheless.
Thanks for clarifying! We always have an expert view section in the report, and often consult animal science specialists, but it is possible we missed something. Could you tell me where specifically we made a mistake regarding animal science that could have changed the recommendation? I want to look into it, to fact-check it, and if it is right not to make this mistake in the future.
2. CE’s charities working on animal welfare have mostly not been very good, and listening to external feedback prior to launching them would have told them this would happen.
[...] doesn’t do CE’s original proposed idea anymore
On the point of the charities not doing CE’s originally proposed idea anymore, I want to clarify that we don’t see charities tweaking an idea as a failure but rather as the expected course of action we encourage. We are aware of the limitations of desktop research (however in-depth), and we encourage organizations to quickly update based on country visits, interactions with stakeholders, and pilot programs they run. There are just some informations that a researcher wouldn’t be able to get, and they need input from someone working on the ground. For example, when Rethink Priorities was writing their report on shrimp welfare, they consulted SWP extensively to gain that perspective. Because CE charities operate in extremely neglected cause areas, there is often no other “implementer” our research team can rely on. Therefore, organizations are usually expected to change the idea as they learn in their first months of operations. I see this as a success in ingraining the values of changing one’s mind in the face of new evidence, seeking this evidence, and making good decisions on the side of co-founders with the support of their CE mentors, and we are happy when we see it happen.
There is a complex trade-off to be made when balancing the learning value from more in-depth desktop research vs. more time spent on learning as one implements, and I don’t think CE always gets it right, but the latter perspective is often misunderstood and underappreciated in the EA space.
Regarding charities specifically, in general, we expect about a 2⁄5 “hit rate” (rarely because of the broad idea being bad, more often because the implementation is challenging for one reason or another), and many people, including external charity evaluators and funders, have a different assessment of some of the charities you list. That being said, if you have any specific feedback about the incubated organization’s strategies or ideas, please reach out to them. As you mentioned, they are open to hearing input and feedback. Similarly, if you have specific suggestions about how CE can improve its recommendations, please get in touch with our Director of Research at sam@charityentrepreneurship.com; we appreciate specific feedback and conversation about how we can improve. Thank you for your support of multiple CE charities so far!
Hi Cecilia!
We offer up to 2,000 USD per month for the duration of the program.
The amount may vary from person to person, with some participants choosing not to take a stipend (e.g., those who would take paid time off from work to attend the program) to others taking the maximum amount (because they have to quit their job to attend).
If you think that amount would not be sufficient to cover your cost of living, please contact us, and we can discuss this on a case-by-case basis.
I agree (of course ;) ), and that’s what we’ve noticed as well. Particularly, there are some crucial research skills that are not being taught elsewhere but are commonly used in EA/when one aims to have a significant impact. For example, prioritization research, calculations of cost-effectiveness at different levels of depth, issues of moral weights, etc. We aim to address this gap as well as provide training in generalizable research skills for example literature reviews. If you know people, who are interested in such a training program, feel free to send them information about it. We would love to see applications from them.
We are still finalizing the list as some ideas come from CE (promising ideas we didn’t have the capacity to research in their respective years), but others will come from foundations interested in research that could affect their decisions. Some of the ideas will also be developed during the program as part of learning how to do idea prioritization, and some may come from other partner organizations.
If it helps, we expect that ideas may come from many cause areas CE is focusing on such as global health and development, biosecurity/health security, governance and health governance, farmed animal welfare. In the majority of those areas, we have a mixture of direct delivery, policy, and meta ideas. But the ideas may go beyond that as well depending on input from partner organizations and foundations.
Thanks Shakeel! I think a big value of the program would come from applying learned skills to practical projects and getting a lot of feedback and guidance from expert researchers when doing so. With those sorts of skills, it is best to lean into learning by doing. That makes it somewhat harder to make “public goods” material that will bring a similar amount of value to the program.
That being said, we are planning to write a research handbook similar to the one we have for the charity incubation program. We probably won’t publish version 1 developed for the upcoming program in October, but we hope to publish version 2 made next year. :)
Thanks Vaidehi! We have established two Theories of Change (ToCs): an initial ToC for the first, pilot program and another for the program’s long-term implementation. With the ambitious goal of piloting this program this year, our priority is to ensure its high value before any potential scaling up, hence the difference between our short-term and long-term ToCs. We are focusing on getting the program up and running, but I will be happy to share the ToC diagram once we are done with the outreach and vetting sprint.
About the expected roles, our curriculum for the upcoming October program is designed to prepare participants for roles in a) grantmaking organizations, b) direct charities in need of research staff, and c) research or evaluation organizations. Training will encompass generalizable research skills, intervention prioritization research, in-depth exploration of specific problem areas and potential interventions within them, as well as conducting external evaluations of charities.
In the longer term, we may expand the variety of research career tracks available.
We also want the research conducted during the program to be immediately applicable and informative for organizational decision-making. So training is not the only output in our ToC, and the other one is producing and disseminating decision-relevant intervention reports and charity evaluations.
I also wanted to quickly check it out while in the office. I played the first 1.5 minutes and it already moved me to tears. I’ll have to wait until I get home to watch the whole movie. Thank you soo much Aaron for sharing it!
Indeed—you can do as Lorenzo suggested. :)
Thank you, Emre!
EAGxIndia was wonderful! I went to many conferences over the last couple of years, and I have to say that EAGx in India had the warmest, kindest and most comfortable, and welcoming vibe; and all of that on top of being impactful. Huge congratulations and gratitude toward the organizers! :)
Unfortunately, it is not possible to apply that to past applicants as it relays on filling out the assessment that we only created and implemented now. However, If you are interested, you can apply to the program now (the deadline is on the 10th of November), fill out the R1 test and just make a note that you want to get feedback. We wouldn’t like people to do it massively as it really makes our job harder to know who is actually applying vs who is there just for the assessment but in a couple of cases, it should be fine. :) Another thing that we do is compile reasons why someone got far and yet hasn’t been selected and then make a post about it—it is not individualized but aggregates the most common reasons and we’ve heard many people found it helpful. Here and here are the posts if you’re interested. We will keep exploring other options as well. Thanks!
Needless to say, when you’re surveying 7 people they have reason to be concerned that they’ll be identifiable from their responses. I will be abstaining from providing details shared with me from the past charity starter including their main concerns because I’m not in a position to judge if they’d be identifiable from that, but would encourage you to collect such feedback. I think it’s safe enough to say that they weren’t from the most recent cohort
That makes sense—if you could encourage them to reach out to me we would be extremely interested in their feedback. I agree that such feedback is very valuable. We collect feedback throughout and immediately after the program from everyone who participated. This is the main way we improve the program for future years. It’s not perfect, nothing is, but we are always trying to get better.
Re non discriminatory hiring, I don’t think the ‘we have neurodivergent staff’ angle is the way to go (ala ‘my best friend is black’ response) - you might just be selecting for neurodivergence in the same direction.
Sorry if I wasn’t clear…. what I was trying to say is that if we have a fairly neurodiverse staff and groups of participants, both of whom had to go through the hiring process, that is evidence that the process doesn’t systematically filter them out.
For that reason, there is merit in say deferring to a resume and work experience which can illustrate that sometimes. Other times, selecting people more generously then ensuring you have good enough training systems to develop them up to do a good job is what you should focus on—suddenly apparent talent bottlenecks looks like a training capability skills gap.
We do put some weight in CVs and work experience and we also think it’s important to look for additional data. We feel that deferring solely to resumes or work experience could bias against some candidates. For example, younger or candidates who have had fewer opportunities in life may otherwise have very good predispositions. As we often say, we look for potential. We think that if we judged based primarily on CVs, that would filter out a lot of people who could do amazing work in the future, but just haven’t had the chance to prove themselves yet.
Ultimately we want to find great people and help them do great things. We’re definitely not perfect and both the individuals in our team and our processes can, and we hope, will, continue to improve. We’re always open to updating and if there are evidence-based practices out there, that we can learn from or adopt, we’re all ears. Again please send them to judith@charityentrepreneurship.com
Hi Maith! We implemented a new system that gives every applicant automated feedback in the form of a short report. That happens after the first round of application. This feedback will be automatically generated based on the responses you gave in the assessment. We are still figuring out how we can give feedback to applicants at the later stages of the application, but basically this system ensures that everyone who applies gets some info on how they scored on each trait, what that means, and occasionally gives some guidance on how to improve those as well.
We have had founders from outside the EA community. “EA-ness” isn’t something we explicitly filter for, rather we look for traits such as being impact-focused, ambitiously altruistic, having strong epistemics etc. Those traits are, on average, more common in EA candidates, but if someone who is not part of the EA movement has those traits they will do well in our process. The only exception was when we had a year on EA meta charities, then being involved and knowledgeable about EA was necessary.
The people that Steve refers to in the quoted sentence are more “spammy” people- those who just seem to apply to every job, or who obviously don’t know what we do, eg., they think we are a grantmaker and they want to get funding for their very non-effective project.
So in your example of people from the international development community, if they have all the traits that matter but are just not a part of the EA community, I think they would make great candidates.
Hi! Thanks for the feedback and for sharing your concerns. I agree that it is a good idea to get a sense of the program from past incubatees. Our incubatees often attend EAGs, so that is a good opportunity and we always encourage that. That being said, I’m surprised by what you say. To explain why, I copied results from a survey conducted after the most recent program in 2022:
Charity Incubation Program:
Foundation Program:
Because of the above, and that it requires a 2 month initial commitment with no guaranteed future, for which the opportunity cost for someone building career capital or in full time employment is high, I have not applied.
We try to help participants as much as possible—we offer a financial stipend not only during the program, but also a bit before and after, so it is not a financial burden for them. The majority of people start a charity and receive seed funding after the program. In the most recent program, all but one person started a charity and received generous seed grants. The one person who didn’t was offered a job at CE and will join the next program again. In previous years, the participants found that they have many more EA job opportunities and better career prospects after the program than before—often, they have multiple job offers from EA-aligned organizations that they didn’t have before. I think it is different if someone wants to come back to the corporate world or work in a non-EA institution, where I think something like participation in the CE IP will be a weaker signal. If you have further suggestions on how we can support people post program, let us know.
Finally, the application process is rather rigid and does not seem open to outliers, neurodiverse applicants (relative to those who apply) who might not fit your mold but might otherwise be good founders.
I agree that our application process is rather rigid, we have been optimizing it over the years. When it comes to neurodivergent candidates, a couple of members of the CE team are neurodivergent, including staff in leadership positions and in the program vetting team. We have had a couple of participants in the program who were non-neurotypical as well. I would be extremely surprised if the process systematically filtered out people based on their neurodiversity. However, if there are any adjustments we can make to the process to accommodate individual applicant needs that emerge from their neurotype, we are happy to do it. In that case, just email Judith, our vetting specialist and she will discuss accommodating the process, her email is judith@charityentreprenuership.com
Unfortunately, we don’t have the data compiled yet in a format that is easy to share, but I will put that on our list of things to publish in the future. But a couple of specific indicators:
There is no significant correlation between age of the participant/experience and charity success.
When we surveyed incubatees who had prior experience to establish how much they apply/use their prior knowledge vs how much they apply what they learned during the program or during running their charity, they say much more often that they regularly apply the charity-specific learning.
We also found that “experts”—people with significant experience and credentials in a given cause area/charity idea, do not systematically outperform people who are new to the field but have other traits (e.g., being entrepreneurial or super impact-focused in the EA sense).
And there is also a practical element to it—often EA-type charities are very, very specific (because of how the impact of charity ideas is distributed) and therefore there is no experience/background you can get that will be as relevant to your organization as actually running the organization, e.g., there is no degree in shrimp welfare in Vietnamese or Indian farming systems :P If you are curious about that case in particular, Andres, Co-founder of Shrimp Welfare Project recently interviewed with Rob and talked about his experience.
I agree with Rob that forgone impact is also an additional (very important) benefit of starting the organization earlier.
But I’d expect people who start a charity earlier rather than seeking additional training first to be systematically different — to start with they’re evidently more confident about their prospects, and that may be an indicator of higher underlying competence or enthusiasm
Yes, I think that’s plausible, but we haven’t found a significant correlation between confidence and charity success either. So even though higher confidence could contribute to the decision to start earlier, it will not necessarily show in charity outcomes. That being said, I also expect there to be a difference between people who start a charity without any support vs. the applicant to the CE program who knows that they will receive training, mentorship etc., and therefore, smoothing out the distribution of confidence in our sample, so the applicant-traits effect may be stronger in independent founders.
We would be excited about people creating another CE. We think that creating a more effective organization is extremely impactful and would be happy to see more organizations working on it. In those few efforts that have been tried by the EA community, we were happy to advise and share our knowledge, and we will be happy to do it for others.
To my knowledge, there is no EA-aligned nonprofit incubator that provides comprehensive support similar to CE. The closest was Longtermist Entrepreneurship (LE) Project, but they discontinued after a year of scoping. They wrote a fantastic post about their learning. A couple of other organizations that offer some support:
FTX offers funding for new organizations, and EA Funds also often seed new projects, I suspect that both founders could also help with advice and connections to relevant people,
Sometimes they are efforts to incubate specific organizations, e.g. civilizational refuges (SHELTER weekend announcement and report from the retreat),
Rethink Priorities and Effective Ventures are offering operational support for new projects.
Outside EA there is also Skoll which offers support for social enterprises and FastForward for tech nonprofits, but I have mixed views about them as they are clearly not EA-focuses and I don’t know enough about the incubation part. In global health and development, there is D-Prize, but they mostly offer funding and not an incubation.
There is also a for-profit & social enterprise route and here you will find tons of incubators, although usually outside of the EA sphere. I personally think that Entrepreneur First is great and we learned from them when we were just starting up. I also know of some EA folks scoping opportunities to start EA for-profit incubator.
We once gave a talk titled: “Impactful opportunities around and adjacent to charity entrepreneurship” you can watch the recording here.
Thanks! Can you tell me more about why you think improving dissolved oxygen is not a good idea? I still consider poor dissolved oxygen to be a major welfare problem for fish in the setting where the charity is expected to operate, and improving it through various means (assuming we also keep stocking density constant or decreasing it) would be good for their welfare. This has been validated in the field by FWI in this assessment and studied by others, so I’m a bit surprised. Unless you are referring to specific interventions to improve dissolved oxygen, of which I have high uncertainty about their cost-effectiveness.
And about the report you link, I broadly agree and have written about it below.