AMA: Leah Edgerton, Executive Director of Animal Charity Evaluators
Note: Aaron Gertler, a Forum moderator, is posting this with Leah’s account. (That’s why the post is written in the third person.)
This is a Virtual EA Global AMA: several people will be posting AMAs on the Forum, then recording their answers in videos that will be broadcast at the Virtual EA Global event this weekend.
Leah will either respond in a video or with written answers. For now, I recommend trying to post questions by 2:00 pm PDT on March 18th (Wednesday), which is when Leah will record her video.
About Leah
Leah has been involved in the effective altruism community since 2011 and has been an animal advocate her whole life.
From 2015–2017, she was an integral part of ACE’s communications team, building up their social media channels, growing the reach of their email list, and helping to plan and launch their website redesign.
From 2017–2019, she worked at ProVeg International, one of ACE’s Standout Charities. In her role as ProVeg’s Strategy and Internationalization Manager, she conceptualized and grew the China Programme from scratch and coordinated with external academics to conduct experimental research on meat reduction interventions. She also worked with the executive team to support ProVeg’s strategic planning, international growth strategy, and internal communication systems.
She returned to ACE’s team as Executive Director in February 2019.
What implications do you think longtermism has, if any, for the work that Animal Charity Evaluators does?
Hey Peter! Thanks for your question! Historically we’ve not engaged too much with longtermism, and so we are yet to make any changes to our programming as a result. It is becoming increasingly clear that this is a cause area that deserves more of our time than we have given it. Recently we have started two projects that we hope will result in blog posts exploring how EAA overlaps with longtermism, hopefully with useful takeaways for those working in each cause area, so keep an eye out!
That’s awesome!
What do you think of the state of evidence and research in the EAA movement now, and how it’s changed over time?
Should EAA be using more sophisticated techniques in causal inference from observational data? Is there data out there we can use already for this? I have in mind Humane League Labs’ upcoming study on cage-free campaigns and analyses of ballot initiatives in California. Can we do the same with attitudes or animal product consumption in response to other interventions, e.g. protests?
Do you think our allocation between narrower interventions and animal movement growth is right, both in terms of resources and research? Should we be going more in one direction over the other? I’m thinking there might be an analogy with Growth and the case against randomista development, by Hillebrandt and Halstead, with movement growth like economic growth, and narrow interventions like randomista (RCT-based) development.
I’m also worried about small sample sizes, as discussed in Gregory Lewis’ post Reality is often underpowered.
We’ve seen a lot of change in our relatively short existence in the movement. Early on, ACE was one of few organizations working to synthesize the existing research, and now we’re in a position where there are lots of organizations doing really great research. This has led to a really positive collaborative spirit, and strengthened the body of research we have available to us in the movement. We are now seeing substantially more funding into research that is of a better quality and using stronger norms (e.g. use of the Open Science Framework). We think there is still room to improve—it can sometimes feel like there is a disconnect between organizations focused on direct work and those that are conducting research, in that it is hard to know what is best to prioritize to serve their interests, and to what extent they are utilizing that research. At ACE, this has caused us to realign our research aims towards conducting research that primarily improves our ability to conduct evaluations and evaluate grants in our Effective Animal Advocacy Fund, as we can be more certain of the impact that research has.
Regarding your question about using more sophisticated techniques in causal inference: Absolutely! We’d love to see more of this type of research.
Regarding your question about the allocation of resources in the movement: My intuition is that our balance is not terrible right now. To give a better answer, I would need to know exactly what the allocation currently is.
Regarding small sample sizes: we are worried about this too. It’s important to consider the full range of evidence when RCTS aren’t an option.
Which views, both ethical and empirical, do you think (should) lead most to prioritizing animals over other EA causes?
I would recommend that those who are trying to better understand their intuitions around which cause areas to prioritize try out 80,000 Hours’ problem quiz.
How much disagreement there is within ACE about which charities to recommend?
The team overseeing recommendation decisions is typically well aligned in making these decisions, and by the time reviews have been drafted for each charity, there are fairly clear levels in the charities’ overall performance. That said, we do of course have disagreements. I think these are more likely to arise when a charity’s status may be changing from the previous year, e.g. an existing charity changing recommendation status or a new charity that might be recommended for the first time. When disagreements arise, we provide as much time as possible to reach an agreement, first in a meeting to understand our points of disagreement, followed up by email discussion if necessary. We then take a final vote to decide. If there is a split decision, the Executive Director makes the final call.
To what extent do you think future reductions in the number of farmed animals will come from advocacy, as opposed to technological advancement (e.g. Beyond Meat)? Do you have a sense of the historical impact of these two approaches?
Hi Ben, that’s a great question. We don’t have a lot of conclusive research on this topic, so what I’m sharing here are just my personal intuitions. I think that food technology will play a major role in the future reduction of farmed animal suffering. However, I wouldn’t consider this an “either/or” question. Many people who work in the food technology space around this issue view their work as advocacy. Additionally, many of the people focused on improving the welfare of animals currently living on factory farms see their work as complementary to the food technology space—some welfare improvements drive up the cost of animal products, perhaps making animal-free alternatives more cost-competitive. Additionally, welfare advocacy raises the importance of animal welfare in the public eye, which is often a strong motivator for those who choose to reduce their animal product consumption.
What kind of research do you think the EAA movement is missing most? Is anyone in the EAA movement (including at ACE) working on it now? What’s the most important EAA research that you’re not aware of really anyone working on?
What kind of research do you think could change ACE’s recommendations most?
Hi Michael, this is a big question! It’s getting a bit late but I want to give you some of my quick thoughts:
Research into reducing fish suffering (scale, promising interventions, tractability, etc.) is needed, and there are a few groups working in this area, such as our Top Charity Albert Schweitzer Foundation and newly-founded organizations Fish Welfare Initiative and Aquatic Life Institute.
Economic research, including research into economic interventions. We have an economist starting on our team in May!
Research into ways we can positively influence the welfare of wild animals, which is something that Wild Animal Initiative is researching (one of our EAA Fund grantees).
Research on estimating the effectiveness of intervention to improve the lives of farmed animals in general. This is something that ACE has worked on, as well as groups like THL Labs, Rethink Priorities, Open Phil, etc. We’d especially be interested in seeing research on how interventions support and interrelate to each other.
I also think it is important for us to look at considerations around longtermism in EAA.
How many resources do you think the EAA movement (and ACE in particular) should invest in animal causes that are less “mainstream”, such as invertebrate welfare or wild animal suffering?
What would convince you that it should be more (or less) of a focus?
Hi Tobias! Thanks for this question. I answered it in the Q&A video recording.
What are some questions regarding EAA which are amenable to being forecasted?
Hello! Thanks for this question. I answered it in the Q&A video recording.
Answers, for reference:
Will corporations stick to their animal welfare commitments?
When will specific animal free food technologies become cost-competitive with their traditional animal counterparts?
Timelines for cultured meat coming to market?
When will technology exist which allows the identification of the sex of a chicken before it hatches? When, if ever, will such a technology be adopted
When, if ever, will the global production and consumption of farmed animals stop growing? When will stop completely?
When will specific countries or states adopt legal protection for animals / farmed animals?
When will EAA organizations have a budget of more than $500 million? $1 billion?
Questions related to the pandemic.
Questions related to the budget of EAA organizations in the immediate future.
Is there a piece of technology which would make your work significantly easier, but which doesn’t exist yet?
Thanks for the question, NunoSempere! Could you clarify whether you are referring to ACE specifically? Or the EAA movement as a whole?
Either / Both
What do you think an EAG attendee is likely getting wrong about ACE?
EAG attendees tend to be fairly familiar with ACE’s work, but if I have to speculate, here are a few things that people may get wrong about ACE:
ACE’s charity recommendations are heavily dependent on our CEEs. Please see this page on our use of cost-effectiveness estimates for more details.
ACE is a “watchdog” organization. I think that this misconception is less prevalent in the EA community, but it is one we encounter fairly often. We recently published a blog post explaining why ACE is not a watchdog organization to help clear up any confusion.
How competitive are the different roles at ACE (and other EAA orgs, if you have an idea), including research, the different internships, etc.? How replaceable do you think people are?
Hi Michael, I’ll respond on behalf of Leah/ACE:
The competition for roles at ACE varies depending on the position. We receive around 10 intern applications every month, so this is by far the most competitive of all positions at ACE. Given our niche area of work, replaceability is a challenge; when we’re looking for someone who has a research background, is interested in effective altruism, and is passionate about animal advocacy, we’re looking at a very narrow group of people.
Speaking more generally, people are replaceable, but the costs—in terms of both time and money—can be very high. It can take months for managers and leadership to recruit, vet, and interview new candidates and up to six months to fully onboard a new hire. During the time we are rehiring for a role, the work assigned to that position is either stagnant or has to be redistributed to other team members who are already operating at capacity. There’s also the opportunity cost for the managers and leadership participating in the process whose time (and corresponding wages) could be better spent on programs and other activities more directly fulfilling our mission.
And while the costs of turnover are high, it’s important to note that the primary reason ACE prioritizes its people is for their inherent value. Each ACE team member is an individual with their own experiences and talents that contribute to achieving our goals. We use job descriptions as guidelines, but welcome the unique insight, ideas, and skills each person brings into their role.
Year after year, we’re able to maintain, and even improve, the quality of evaluations, research, and grant-making for which ACE is internationally known by investing in our people—people with exceptional competencies and the dedication to finding and promoting the most effective ways to help animals.
What do you think are the main bottlenecks and limiting factors in the EAA movement?
Hi Michael! Thanks for this question. I answered it in the Q&A video recording.
Which interventions do you think are the best now? Which less well-studied or new interventions do you think could be competitive with them?
Thanks for this question. I answered it in the Q&A video recording.
In https://animalcharityevaluators.org/transparency/financials/ you claim that
What do you think is the counterfactual of these influenced donations? How much of this $26 million do you think would have been not donated to any animal charities, and how much to less effective animal charities?
We seek to gather data about this question through our annual Donor Survey. Our most recent published data is from 2018, and we are planning to publish the results from our 2019 survey soon.
Do you think that for remote organizations staff retreats are worth the costs and staff time? How confident are you in your opinion?
Hi Saulius! Thanks for this question. I answered it in the Q&A video recording.
What would you say to someone who is undecided between donating to ACE and one of ACE’s top charities?
(note: this does not apply to me but I thought that it may apply to some of the listeners)
Thanks for this question. I answered it in the Q&A video recording.
Do you think that EAA should increase or decrease the proportion of resources that are spent on research rather than action?
My intuition on this is that our movement spends an appropriate percentage of our resources on research, but I’m sure others have different opinions on this question. I’d like to see the total budget of EAA increase dramatically though—both the amounts spent on direct work and the amounts spent on research.
What are the most important factors that determine which charities get recommended?
Note that ACE does write about their criteria here. But I’m interested in which factors are the most impacful. E.g. if charities that seem to be cost-effective often don’t get recommended because of a lack of plans for growth or an unhealthy culture.
How a charity performs on our seven evaluation criteria is the strongest determinant as to which charities receive a recommendation from us. When deciding whether to award a charity a Top or Standout Charity status, the charity’s room for more funding tends to be a deciding factor.
Alot of EAA focus in regards to resource seem to be on more developed countries like USA and UK leaving many other countries neglected despite the fact that they also have high meat consumption e.g. China, India, South Africa etc. What do you think are the most effective ways we can try to expand the movement internationally. Do you think this should be tackled by existing EAA organisations or new localised organisations?
Also thank you for agreeing to do this talk!
Hi Lauren! Thanks for this question. I answered it in the Q&A video recording.
How do you see ACE changing and evolving in the future?
There are a few ways in which we see ACE changing and evolving in the future. Given the number of new research organizations that have entered the EAA space since our founding (yay!), we will likely narrow our focus to make sure we continue to add as much value as possible to the movement and don’t duplicate the efforts of others. In particular, we’ve chosen to narrow the focus of our non-evaluations/grantmaking research this year exclusively to questions that can help better inform our decision-making in those two processes. This is partially because other orgs are doing a great job working on some of the broader foundational questions facing our movement, and partially because we feel most confident in implementation of research that we know we can implement ourselves.
We are also making some changes to the way that we operate internally. This year we will be rolling out a new operating model, based partially on Scrum and Agile project management frameworks, with the goal of streamlining and speeding up our workflows, empowering all of our staff members, and alleviating high workloads for senior staff members. We call our in-house project management style “Scram”...
GiveWell is conducting research in new areas, in hopes that they’ll find charities to recommend in those areas even though this hasn’t happened before.
Is ACE currently doing any work like this—to find new promising areas that will open up more charities to potentially be recommended? If so, what area(s) are being looked at?
What have you changed your mind on recently?
Hi Misha! Thanks for this question. I answered it in the Q&A video recording.
Are you concerned about any negative effects of ACE’s charity recommendations (e.g. over-optimizing, unhealthy competition etc.)? If yes, do you do anything to mitigate these effects?
Thanks for this question. I answered it in the Q&A video recording.
Much of animal welfare initiatives seem to focus on farmed animals. Farmers are experiencing weather extremes, less-predictable seasons, wildfires, and flooding from climate change, which are likely to impact farmed animal welfare. If at all, how does this influence the strategy of the animal welfare movement?
Hi Louis! Thanks so much for your question. I feel that the movement’s strategy should always be influenced by the holistic context in which we operate and current events. One quick example that comes to mind of a project working directly with farmers is Mercy for Animals’ recently launched “Transfarmation” project, which helps support farmers in transitioning out of animal agriculture and into growing crops for plant-based products.
What do you think an EAG attendee is likely getting wrong about animal advocacy? Effective animal advocacy?
Thanks for the questions.
Regarding animal advocacy, some EAG attendees might assume that animal advocacy is all about direct care of injured animals, especially companion animals, and about changing the attitudes and opinions of the general public towards them. However, animal advocacy is a broad and diverse movement, where people carry out varied interventions targeting different animal issues, ranging from lobbying politicians to advocate for farmed animal welfare laws, implementing alternatives to animal products, developing research on how to help wild animals, and building alliances with other social movements.
Regarding effective animal advocacy (EAA), some might think that EAA is all about considering the quantitative short-term impacts of interventions, overlooking the importance of good management practices, and other factors that can indirectly affect the effectiveness of interventions. However, we believe that effective animal advocacy goes beyond a quantitative analysis of effects; it also consists of following good management practices and creating a healthy culture.
We published a blog post a few years ago where we outline some common misconceptions about effective animal advocacy, here is the link if you are interested in reading it.
I sometimes worry that if in a given year charity’s funding significantly depends on whether they are recommended by ACE, it might be difficult for them to make long term plans and hiring. Because of this, I considered donating to charities that ACE has stopped recommending at any given year so that they wouldn’t have to lay off people. Do you think this is a significant concern? (P.S. people other than Leah are also free to answer in a comment).
Rescinding charity recommendations is not something that we take lightly. We have only done this in the case of serious issues that were brought to our attention, and because of this reason, I find it hard to recommend one universal course of action. In some cases, we have recommended that previous donors continue supporting a charity at a lower level in order to prevent the issue you described. In other cases, we have not done so.
In general, we feel that supporting our Top or Standout Charities, Recommended Charity Fund, and Effective Animal Advocacy Fund offer the greatest amount of impact, and that is the advice that I would give to donors who ask.
Most of the animal welfare organizations that I know of which seem unusually effective are somehow related to EA. (E.g. I see staff from ACE’s top charities regularly at EA Global.)
Are there parts of the effective animal advocacy ecosystem which don’t overlap with EA? Do you have a sense for why these parts aren’t involved with EA?
Is there any one thing you think that EAA and AA organisations can focus on or should focus on which would significantly improve the impactfulness of the movement as a whole?
I’d like to see more organizations focus on developing their management, leadership, and governance capacities. I know that your research at Animal Advocacy Careers on skill gaps in the movement identified this as a priority, and it’s something I hear repeatedly from other funders in the movement as well.
The reason this has become such a priority is a side effect of a great problem to have—our movement’s funding has grown significantly in the past several years, and organizations are growing larger than ever and able to take on more ambitious goals than ever before.
How useful is reducing animal farming for reducing pandemic risk?
Question taken from: here
I predict you’re going to see a lot more about this in the coming weeks and months.
Roughly, what percentage of ACE’s staff-time and money is spent on each of the following charity evaluations, foundational research, communications, management, staff retreats and travelling?
This is answered here
In which ways, if any, are the ethical stances which you take counterintuitive to you? Are there ways in which you expect your ethical stances to be counterintuitive to others, and any ways you adjust for that?
While I’ve been working in this space long enough that many of our ethical stances don’t feel counterintuitive to me anymore, I’m sure some of our stances come across that way to mainstream audiences. Farmed animal advocacy is still considered a very niche topic in society at large, even though it’s the primary focus of our work at ACE. Wild animal welfare is another topic that we find challenging to communicate about to mainstream audiences, even though it’s a high priority for many of us in the EAA community (I remember when it felt counterintuitive to me!). We adjust to that by being mindful in our communications to meet our different audiences where they are at, and not assume that everyone has the full context that we do from having been steeped in this topic for years.
What does “Hits-based Giving” look like for animal advocacy?
Should we be focusing more on “work that is more than 90% likely to fail, as long as the overall expected value is high enough?” [1]
If “effectively all the returns are concentrated in a few big winners, and … the best ideas look initially like bad ideas” [2], then how can ACE use evidence in an “epistemically permissive” [3] way to find those big winners?
[1] https://www.openphilanthropy.org/blog/hits-based-giving
[2] http://paulgraham.com/swan.html
[3] https://forum.effectivealtruism.org/posts/bsE5t6qhGC65fEpzN/growth-and-the-case-against-randomista-development#9B2DWc49MRcYYf7j4
Hi Matt! Thanks for this question. I answered it in the Q&A video recording.