Highlights From Our 2023 Reddit AMA

Animal Charity Evaluators29 Nov 2023 11:41 UTC
16 points
1 comment16 min readEA link

In our live Reddit Ask Me Anything (AMA) event, our Programs team answered questions about our new charity recommendations and the processes behind our selections. Below, we’ve rounded up some highlights from the AMA. We hope these questions and answers provide deeper insight into our 2023 charity evaluations and decision-making processes. You can view the full AMA thread here. Thank you to Vegan Hacktivists and the r/​vegan team for hosting this AMA for us.

Do you want to support our highly impactful 2023 Recommended Charities through a single donation? Double your donation here!

Note: Questions and answers have been edited for length and/​or clarity. Links to the original sources on Reddit are provided beneath each response.

→ It’s great to see diverse approaches to animal welfare across the Recommended Charities. Do you intentionally select charities with diverse approaches to animal welfare, or is it a byproduct of the variety of charities doing great work?

At ACE, we aim to help as many animals as possible, and our primary role is to direct funding to charities that are achieving that as effectively as possible. We have a process for prioritizing interventions, which you can learn more about here. However, because animal advocacy is a young movement, there is little data on the effectiveness of different interventions, especially on interventions that have longer-term, systemic theories of change. It’s also important to note that different interventions can be interdependent, with the effectiveness of one intervention depending on the use of others (e.g., the movement may need to promote meat alternatives from both the supply side and the demand side). Therefore, we are intentional in fostering plurality among our Recommended Charities (especially a plurality of promising interventions) while at the same time recommending only those charities we evaluate as being highly effective. So, overall, it’s a bit of both—that is, the diversity of approaches to animal welfare across our Recommended Charities is intentional, and it comes as a byproduct of the variety of charities doing great work.

(Source)

→ How diverse are your recommended charities in terms of racial and gender demographics? Do you know what percentage of the staff (especially leadership) at these charities are BIPOC or women? Is this something you collect data on?

We don’t currently collect data on the individual characteristics (like race, ethnicity, education level, gender, disability status, etc.) of the staff at the charities that we evaluate. In part, this is because we don’t believe we’d be able to do this sensitively and meaningfully given that our evaluated charities cover a range of countries and cultures where demographics are likely to differ for a variety of reasons, not all of which are relevant.

Instead, we seek to incorporate representation, equity, and inclusion into our Organizational Health assessments through other means, such as by asking charities’ leadership which relevant policies and processes they have in place (including written commitments not to tolerate discrimination on the basis of social characteristics, processes to attract diverse candidate pools, and standardized processes for things like hiring and employment termination). In our staff engagement survey, we specifically ask staff whether they agree that their organization has a clear process to address instances of harassment or discrimination at work, as well as asking broader questions around psychological safety in the workplace.

In order to be able to do the most good we can as an organization, we believe it is important to include the ideas of people with varying experiences and expertise. We think that’s necessary to be able to make the best decisions.

(Source)

→ Can you explain more about why New Harvest didn’t get a recommendation? The review page just links to your general criteria.

You can see how we evaluated New Harvest’s work in the spreadsheets linked in their review. We linked the general criteria as additional context for readers to interpret the spreadsheets. Overall, we think that New Harvest’s recent achievements are fairly cost effective relative to the other charities we evaluated, but New Harvest ranked lower on Impact Potential than other charities. Our overall assessment of New Harvest was also based on our general uncertainty about the impact of the field of cellular agriculture, the fact that there’s significant private investment funding in the cell-cultured meat space (so perhaps less of a need for philanthropic funding), and the fact that we already have another Recommended Charity (The Good Food Institute) that works on cell-cultured meat. That said, while New Harvest didn’t receive a recommendation from us this year, we value the work that they do and recognize them as one of the most effective charities in their space.

You can read more about our process for making final recommendation decisions in our 2023 Evaluation Process blog post. When making our final recommendation decisions based on the four evaluation criteria, charities are not compared to all other animal charities in the world; instead, they are compared to the other charities we are evaluating in a given year. In the case of New Harvest, we felt that their scores didn’t align with our final decision, but their Impact Potential and Cost Effectiveness scores were good. Room For More Funding and Organizational Health were also good. Something we considered more this year was that we couldn’t build in a difference between cell-cultured meat and plant-based alternative proteins in our model, so we subjectively adjusted our model in New Harvest’s case.

(Source)

→ Why didn’t Mercy For Animals get recommended this year, and did you evaluate Animal Equality?

We think that Mercy For Animals (MFA) focuses on animal groups, countries, and interventions that we consider high priority. We currently think that Mercy For Animals’ achievements over a 12-month span were slightly less cost effective than those of the other charities we evaluated this year. In particular, we think that other organizations were able to have a higher reach per $ in institutional outreach campaigns. Importantly, this does not mean that we think MFA doesn’t do impactful work, and we recognize them as one of the most impactful organizations in their space. However, we think that the limited funding available in the space currently would have a higher marginal impact if directed to our recommended charities.

Mercy For Animals notes in their review that they think there may be a tradeoff between cost effectiveness and organizational size. They think that larger organizations often engage more effectively with influential global entities, which can come at the expense of increased investment in infrastructure and people support.

We did not evaluate Animal Equality this year. You can see the full list of charities we evaluated in 2023 in this blog post.

(Source)

→ Why was the closure of THL México, and the reasons for it, not included in the evaluation?

Thanks for the question. The closure of THL México was a factor in our evaluation of The Humane League (THL). After THL made us aware of the THL Mexico closure and provided us with a lot of information about the causes and implications of this decision, we held a team discussion to decide whether this should impact our recommendation decisions. We then had a back-and-forth with THL’s leadership to resolve outstanding questions and concerns. Overall, the context that they provided left us without any major concerns about recommending them. I hope you’ll understand that the specifics of this are confidential information that ACE isn’t able to share.

This is summarized in the Organizational Health section of THL’s review: “After our evaluation process had been completed, THL México’s board of directors made the decision to close THL México due to unexpected resource challenges. Following discussion with THL’s leadership about the causes and impact of this decision, we decided that this did not change our decision to recommend them.”

(Source)

EVALUATION PROCESS

→ Does ACE prioritize animal welfare (improving conditions) over animal rights (promoting non-use), or is there a balance between the two in your evaluations?

Thanks for this great question. At ACE, we aim to help as many animals as possible, and our primary role is to direct funding to charities that are achieving that as effectively as possible. We take into consideration all approaches, so we don’t tend to prioritize either animal welfare or animal rights—that is, we are neither solely welfarist nor solely abolitionist. Instead, we look carefully at the programs a charity is working on, the interventions they use, and their theory of change. In our Menu of Interventions, some interventions are focused on a welfare approach (e.g., producer outreach, corporate outreach), and some are focused on animal rights (e.g., vegan outreach). However, when considering programs that are welfare-focused, we prioritize efforts that are a stepping stone to systemic change. (We take into account concerns that welfare-focused work might end up propping up harmful animal industries in the long term.) When we make our final recommendations, we try to strike a reasonable balance between welfarist and animal rights approaches; for example, this year’s recommendations include Shrimp Welfare Project, whose work is welfare-focused, New Roots Institute, whose work is more animal rights-focused, and Wild Animal Initiative, whose work can be both.

Overall, we are anchored in our guiding principles of anti-speciesism and doing the most good we can. We care about animals’ experiences both now and in the future, so we want to support approaches that can help the most animals in both the short and long term. This means striking a balance between welfare-based approaches that are highly tractable in the short term and animal rights approaches that are highly promising in the long term.

(Source)

→ Are there specific regions or countries where ACE focuses its evaluations more, and if so, why? How do you ensure global representation in your recommendations?

The countries or regions where the charity works is a factor we consider in our charity evaluations. This year, we created a country prioritization framework for farmed animal organizations. We considered several proxies for Scale, Neglectedness, and Tractability and scored 196 countries relative to each other. By using this framework, we aim to prioritize countries with relatively large animal agriculture industries (or where the animal agriculture industry is projected to become large in the near future), few other charities engaged in similar work, and where animal advocacy is likely to be feasible and have a lasting impact. Additionally, we considered proxies for a country’s Global Influence to account for possible spillover effects in other countries. For more details on how we score and prioritize countries, see the country-relative scores spreadsheet.

Additionally, when selecting charities to evaluate this year, we sought to include a broad range of charities working in high-priority and unrepresented countries. However, our capacity only allows us to evaluate a few charities in total, and therefore, we are aware that this is a limitation of our process. We are considering how to improve this process in future evaluations to ensure that they are globally representative. This aim is also supported by our Movement Grants program, which specifically prioritizes projects in underrepresented regions.

(Source)

→ Where do you start looking for organizations to evaluate, and do you follow their work for a while before choosing which ones to recommend?

We have evaluated some of our Recommended Charities repeatedly over many years, but we also evaluated several charities for the first time this year. We want to make sure that we evaluate a diverse group of charities, including newer organizations and organizations from geographical regions we may be less familiar with, so we don’t want to focus only on organizations whose work we have followed for a while.

During the charity evaluation process, we analyze charities’ achievements over a period of 12 months and look at their financial information from 2020 onwards, so we have a good idea of what each charity has spent its funds on in the recent past before making recommendation decisions. You can learn more about our charity evaluation process here.

(Source)

→ How do you balance risk vs. potential impact when supporting big, well-established charities alongside grassroots teams/​organizations and innovative approaches? Also, is it a fair assumption that more support goes to welfare-focused initiatives than abolition-focused ones? If so, is that more likely a reflection of welfare-focused organizations’ tendency to have better metrics/​data and demonstrable “wins” or because welfare initiatives are easier to sell when fundraising?

The risk vs. impact question highlights the different roles and theories of change of ACE’s two programs: Charity Evaluations and Movement Grants. In our Charity Evaluations program, we prioritize potential impact (and actual achievements) over risk. To be evaluated, charities need to have existed for at least two years and have an operating budget of at least $200,000 USD to be eligible for evaluation. To make our recommendation decisions, we lean on available empirical evidence about the effectiveness of interventions, and we have internal processes for prioritizing animal groups, countries, and interventions. This is because we want to be confident that the charities we recommend to donors will use additional funding effectively.

However, we know that (i) the animal advocacy movement is young, (ii) there is a lack of empirical evidence on the effectiveness of interventions, and (iii) there are highly neglected interventions and global regions where additional donations might have an outsized impact if we take small risks and provide funding to support them. That’s where our Movement Grants Program fits. While we make evidence-based grant decisions using the similar prioritization methods we use for our Charity Evaluation Program, we are also more risk-tolerant and aim to support more grassroots efforts. As such, we’ll fund individuals as well as newly established organizations.

At ACE, we aim to help as many animals as possible, and our primary role is to direct funding to charities that are achieving that as effectively as possible. In both our Charity Evaluation program and our Movement Grants program, we take all approaches into consideration, so we don’t tend to prioritize either animal welfare or animal rights—that is, we are neither solely welfarist nor solely abolitionist. Instead, we look carefully at the programs a charity is working on, the interventions they use, and their theory of change. In our Menu of Interventions, some interventions are focused on a welfare approach (e.g., producer outreach, corporate outreach) and some are focused on animal rights (e.g., vegan outreach).

When considering programs that are welfare-focused, we prioritize efforts that are a stepping stone to systemic change. (We take into account concerns that welfare-focused work might end up propping up harmful animal industries in the long term.) When we make our final recommendations or grant decisions, we try to strike a reasonable balance between welfarist and animal rights approaches; for example, this year’s recommendations include Shrimp Welfare Project, whose work is welfare-focused, New Roots Institute, whose work is more animal rights-focused, and Wild Animal Initiative, whose work can be both.

While you make a great point about welfare-based approaches being more “measurable” in the short term (that is, easier to estimate the number of animals’ lives helped/​saved per dollar) and possibly easier to fundraise for, we take a balanced approach because there is a risk that focusing only on what’s immediately and easily measurable will leave a lot of impact on the table. We don’t want to fall prey to measurability bias because the animal advocacy movement is relatively young and woefully underfunded, so to help the most animals as much as possible, we need a diverse range of approaches, even those that are harder to measure. At ACE, we also want to support charities that work on longer-term systemic change for animals, even if it’s hard to measure the impact in the short term.

Overall, we are anchored in our guiding principles of anti-speciesism and doing the most good we can. We care about animals’ experiences both now and in the future, so we want to support approaches that can help the most animals in both the short and long term. This means striking a balance between welfare-based approaches that are highly tractable in the short term and animal rights approaches that are highly promising in the long term.

(Source)

→ Have you ever come across charities that don’t want to be evaluated or faced government resistance to evaluating a charity? Also, what are some general steps/​templates to evaluate an animal charity?

To the first one: Yes, some charities do decline our invitation to be evaluated. This can be because they feel their organization is too young or they’re going through a transition period, so don’t feel well established enough to be evaluated; they’re too busy to make the time commitment that an evaluation entails; they don’t consider themselves an animal charity or want to be framed as such; they disagree with our evaluation criteria, methodology, and/​or philosophy; or they don’t support our decision to evaluate charities relative to one another. And yes, in some cases, nonprofit organizations’ need to maintain strong relations with their country’s government (and frame their public image accordingly) can mean that they view it as risky to be associated with a U.S.-based animal advocacy organization like ACE.

To the second question: From our experience, evaluating an animal charity boils down to answering four fundamental questions:

How promising does their work seem? For example, do they work on animal groups that are suffering in especially large numbers or are currently very underrepresented? Do they work in countries where there are a lot of animals in need of help, with few organizations working to help them, and where there seems to be strong potential for impactful animal advocacy work? We call this Impact Potential.)

How much have they already achieved? What projects have they completed, and how much did these cost? How impactful do we think these achievements have been, relative to cost? (We call this Cost Effectiveness.)

If they are recommended, would they be able to use the additional donations effectively? How much extra money does the charity say they could effectively use over the next two years to help animals? Does this seem realistic based on their arguments and their past spending? (We call this Room For More Funding.)

Does anything seem risky about the way they run their organization? Do their staff report issues or seem unhappy and unmotivated? Are there major gaps in their workplace policies? Does their leadership seem unwilling to take potential issues seriously? (We call this Organizational Health.)

In practice, answering these questions involves asking charities for a lot of information about the work they’ve done, the work they plan to do, and the way they run their organization. We ask for this information from the charity’s leadership team and also send a survey to their staff to check how happy and engaged they seem. We then use a variety of methods to score the information they provide us with to make recommendation decisions. You can read more about those methods in our charity reviews.

(Source)

→ Do you have any metrics about how many animal lives are saved or changed per dollar amount (e.g., every $10, $100, $1,000), similar to GiveWell?

We have been thinking a fair bit about coming up with quantified estimates of impact on animals or the number of animals saved per $. We moved away from such estimates in 2019 due to the high uncertainties involved. However, we see the value in using quantitative information for transparency, ease of interpretation, and to increase objectivity. Since 2022, we have been moving toward including more quantitative information in our cost-effectiveness analysis. In 2023, we carefully considered whether a fully quantitative model for estimating the impact on animals per dollar was possible. We decided against it for several reasons.

We are in a very different situation compared to GiveWell because the empirical evidence we have available for effective animal advocacy is much more sparse than what we know about global health interventions. It’s also plausible to assume that animal advocacy interventions are interdependent to an extent that many global health interventions are not (e.g., you may need both individual- and institutional-focused interventions to create progress for animals).

Our charities also employ a wide range of interventions, and all funding we direct to recommended charities is unrestricted, so even if we were to quantify the impact of some interventions, it would be difficult to generalize that to the full scope of the work of an organization. While quantifying impact is more straightforward for some interventions (e.g., corporate outreach for welfare improvements), the charities we evaluate employ a wide range of interventions (our Menu of Interventions currently covers 26 intervention types), many of which are much harder to quantify, especially those aiming for more systemic or long-term change (e.g., research or government outreach). Fully quantified cost-effectiveness estimates also often necessarily ignore important factors that are hard to quantify and are sometimes arbitrary in what they include and exclude.

We wanted to be able to compare charities using all intervention types, and therefore, we decided on a weighted factor model that allowed us to combine quantitative information with more subjective qualitative judgments. In the Cost Effectiveness section of our charity reviews, you can see the metrics we used to compare charities. This includes, for example, the number of individuals reached per $ in education campaigns or through skill and network building. Some charities also provided their own estimates of the number of animals affected by their achievements, and these are included in footnotes where applicable. For example, Shrimp Welfare Project estimates that the welfare commitments they obtained from shrimp producers affected 1,708,928,571 shrimps.

We are constantly updating and improving our methods, and we will be revisiting this decision and carefully considering what is possible for the next (and future) round of evaluations.

(Source)

ANIMAL ADVOCACY

→ Wild animal suffering seems like a terribly urgent issue, however, some advocates make the case that we should solve farm animal suffering first. What is your take on this, and what immediate steps should we take as a movement and as individual advocates to help wild animals?

Thanks for this question! There are strong arguments in favor of prioritizing both wild animals and farmed animals. Both cause areas are high-priority for ACE. Wild animal suffering is one of our high-priority cause areas, especially because of its great scale and high neglectedness. However, because of the young state of the wild animal suffering movement and how little we know about how to help wild animals in the most impactful ways, we consider this cause area as having low tractability in general.

One of our Recommended Charities is Wild Animal Initiative, which is doing work to build an academic field that will lay the groundwork for developing interventions to help wild animals. We believe that this foundational work can be highly impactful in the medium and longer term. Additionally, considering the scope of individuals, early and large investment in a new cause area could have a significant impact. The strengthening of the academic field and building a healthy, respected community is essential in the early development stages to prevent catastrophic approaches and/​or reputational harm, and to build community infrastructure. Wild Animal Initiative’s Research page can give you an idea of some of the top-priority research topics in this space, such as the potential benefits of contraception as a means to mitigate wild animal suffering.

Currently, we are aware of only a handful of organizations working on reducing wild animal suffering, and we would like to see and evaluate more organizations working on this cause area in the future.

(Source)

→ What do you think the role of education is within animal advocacy? What do you think advocates whose work centers around public education focus on and why?

Thanks for this question! We think education is a promising intervention and could be an important tool for raising awareness among young people, which could have downstream effects on demand for animal products, engagement in animal advocacy, and policy. There is some evidence that educating students on the environmental effects of animal agriculture and the health benefits of meat reduction can reduce meat consumption and increase purchases of plant-based meals (see here and here). Other studies have found, for example, that ethics classes on moral decision-making can reduce meat purchases and increase the perception that factory farms are unethical.

However, we need a lot more research. A lot of the existing research measures changes in the short term, so there is still uncertainty around how long-lasting changes in attitudes and behavior are. Research also largely focuses on meat reduction and attitudes toward farming, so it is less clear how education can shape, say, political engagement on behalf of animals. There are also a lot of unanswered questions about what makes education more or less effective (e.g., age groups, content, and modes of teaching).

Two great charities among our 2023 Recommended Charities that you can look into and support are New Roots Institute and Faunalytics. New Roots Institute builds capacity in the animal advocacy movement through interactive high school and college lessons on industrial animal agriculture and their Leadership Fellowship Program. Faunalytics conducts research to identify the most promising interventions in animal advocacy.

→ How urgent is it that we focus on insect farming as an issue? If it is urgent, what do you think we can and should do about it?

At ACE, we prioritize causes based on Scalability, Neglectedness, and Tractability. While billions of land animals are killed for food each year worldwide, when you include insects, this number jumps to trillions of animals. (For more detailed figures, as well as some useful sources and some of the considerations around insect sentience, you can read our June 2022 research brief on this.) Despite them being killed in such large numbers, there is currently little to no consideration given to the welfare of insects being farmed. (That said, there’s some interesting evidence that insect farmers do feel uncomfortable about some elements of insect farming and give at least some consideration to their welfare on a personal level.)

When prioritizing animal groups, we also take into account their likely capacity for suffering and wellbeing. As you would expect, this is extremely challenging. This year, we estimated priority scores for different animal groups based in part on the recent Moral Weights Project carried out by Rethink Priorities. (You can see our scoring sheet here.) While we currently believe that the capacity of insects to experience suffering and wellbeing is likely to be significantly lower than that of other farmed animals, we are open to being updated on this, given how little research there has been to date on farmed insect sentience. Given this, alongside the huge scale of the insect farming industry and the very small number of people working on insect welfare, we think there is a strong case for insect welfare being a high-priority topic and hope to see more organizations exploring tractable interventions in this space.

(Source)

→ Last year, you said that you consider protests to be “a very low-priority intervention based on relevant research.” Is this still your position on the subject, and if so, what research is this based on?

We aim to regularly update ACE’s stance on the effectiveness of different types of interventions based on relevant research. This year, we categorized the interventions animal advocacy charities use into 26 types and the main outcomes they work toward into eight types. We scored 77 combinations of interventions and expected outcomes. See more details on how we did this in the intervention-relative scores spreadsheet. Overall, we rated protests as moderate-priority this year.

We base these scores on the relevant research available. In the case of protests, the evidence is mixed. For example, ACE’s 2018 protest intervention report suggested that protests can positively influence public opinion, industry, alliances, policy, and movement capacity, at least in some contexts. A 2022 report by Social Change Lab also found favorable conclusions about protests. However, a 2020 empirical study suggests protests may reduce support for a social movement if they are perceived as harmful or highly disruptive. Additionally, a 2022 report by Faunalytics recommends against both non-disruptive and disruptive protests as animal advocacy interventions. Similarly, a 2023 study found that reading about a vegan protest, irrespective of how disruptive it was, led to worse attitudes toward vegans, and greater defense of meat consumption than reading about a control protest.

As new research is published, we will continue to update our understanding of the effectiveness of different interventions.

(Source)

→ Do you have any stats on how many total chickens are now cage-free due to the Effective Animal Advocacy movement?

It’s difficult to estimate the total number of chickens affected by the EAA movement, but here are some stats from Open Philanthropy and some of our recommended charities:

  • In 2020, ACE Recommended Charity The Humane League estimated that since 2007, the number of cage-free hens has increased by 96 million in the U.S.

  • Open Philanthropy estimates that 165 million more hens are cage-free in Europe and the U.S. today than were a decade ago, thanks to corporate outreach campaigns.

  • The Open Wing Alliance and other animal charities have globally achieved more than 2,500 cage-free commitments, and Chicken Watch tracks their progress toward their targets here. Recommended Charity Sinergia Animal similarly tracks the progress toward cage-free targets in Latin America and Asia here.

(Source)

Animal Charity Evaluators29 Nov 2023 11:41 UTC
16 points
1 comment16 min readEA link