Principal — Good Structures
I previously co-founded and served as Executive Director at Wild Animal Initiative, and was the COO of Rethink Priorities from 2020 to 2024.
abrahamrowe
Karma: 5,230
From this post, it seems like you’re trying to calculate historic cost-effectiveness and rate charities exclusively on that (since you haven’t published an evaluation of an animal charity yet I could be wrong here though). My understanding of what ACE is trying to do with its evaluations as a whole is identify where marginal dollars might be most useful for animal advocacy, and move money from less effective opportunities to those. Cost-effectiveness might be one component of that, but is far from the only one (e.g. intervention scalability might matter, having a diversity of types of opportunities to appeal to different donors, etc.). It’s pretty easy to imagine scenarios where you wouldn’t prefer to only look at cost-effectiveness of individual charities when making recommendation, even if that’s what matters in the end. It’s also easy to imagine scenarios where recommending less effective opportunities leads to better outcomes to animals—maybe installing shrimp stunners is super effective, but only some donors will give to it. Maybe it can only scale to a few M per year but you influence more money than that. Depending on your circumstances, a lot more than cost-effectiveness of specific interventions matters for making the most effective recommendations.
My understanding is also that ACE doesn’t see EAs as its primary audience (but I’m less certain about this). This is a reason I’m excited about your project—seems nice to have “very EA” evaluations of charities in addition to ACE’s. But, I also imagine it would be hard to get charities to participate in your evaluation process if you don’t run the evaluations by them in advance, which could make it hard for you to get information to do what you’re trying to do, unless you rely on the information ACE collects, which then puts you in an awkward position of making a strong argument against an organization you might need to conduct evaluations.
My understanding is ACE has tried to do something that’s just cost-effectiveness analysis in the past (they used to give probability distributions for how many animals were helped, for example). But it’s really difficult to do confidently for animal issues, and that’s part of the reason it’s only a portion of the whole picture (along with other factors like I mention above).
…we have reviewed 5 of ACE’s “Top 11 Animal Charities to Donate to in 2024” and only one of them (Shrimp Welfare Project) appears to be an effective charity for helping animals. ACE’s poor evaluation process leads to ineffective charities receiving recommendations, and many animals are suffering as a result.
I understand these are forthcoming, but no evidence is provided for this entire part—part of the reason I pushed on this is I think seeing your alternative evaluations would be very helpful for interpreting the strength of the critique of ACE. Without seeing them, I can’t evaluate the latter half of the quoted text. And in my eyes, if these are similar to the evaluation here of LIC, it’s pretty far from demonstrating that ineffective charities are receiving recommendations, etc. And, given that you’ve only evaluated <50% of their charities so far, it seems preemptive to make the overall claim. I think the overall claim is very possibly true, but again, I think to make the argument that animals are directly suffering as a result of this, you’d have to demonstrate that those charities are worse than other donation options, that donors would give to the better options, etc.
Thanks! My wording in the above message was imprecise, but I mean something like farmed vertebrates. SWP is probably among the two most important things to fund, in my opinion.
Basically I think the size of good opportunities in farmed animal advocacy is smaller than OpenPhil’s grantmaking budget and there are few scalable interventions, though I don’t think I want to go into most the reasons publicly. Given that they’ve stopped funding many of what I believe are more cost-effective projects, and that EA donors are basically the only people willing to fund those, EA donors should be mostly inclined to fund things OpenPhil can’t fund instead.
So some combination of 1+2 (for farmed vertebrates) + other factors
(I responded privately to this but wrote up some related reflections a while ago here).
Thank you for reading some of the article. I hope that you find some time to read the rest.
To be clear, I read the whole thing—I meant that I think the fact that a pretty important issue jumped out to me within a few minutes of starting reading struck me as a reason that getting feedback from ACE seems really important.
Did you ask ACE to review this before publishing?
I really think you should! I also really think you should ask for feedback from other people who have done charity evaluations, and the charities you evaluate. You should definitely still publish them, but they’ll be better critiques for having engaged with the best case for the thing you’re critiquing!
What do you mean by conceptualizing funds? In this hypothetical, they simply spend $200k less on the lawsuit. LIC did not spend their entire budget, and charities oftentimes do not. Under ACE’s methodology, LIC’s cost-effectiveness would improve if they spent $200k less and achieved the exact same total outcomes as a charity. The calculations we’ve done are 100% objective, and if you can find an error that we made, please let us know. You can find those calculations here:
Yep, this seems right, but it’s also the case that if they did something else with that funding, the effectiveness of that action would be rated much more highly, which also seems correct. I think the issues you point to are interesting, but they strike me as intentional decisions, which ACE may have internal views on, and for which I think getting their feedback might be really important. You are correct about a mathematical fact, but both you and ACE seem to have different goals (calculating historic cost-effectiveness vs marginal impact of future dollars), and there are assumptions underlying your analysis that if changed, might change the output.
What assumptions are you referring to?
I meant ACE’s assumptions—I thought your post raised some really good questions. They are issues that if I saw, I’d email to ACE and ask why they made the choices they made, then choose whether or not to publicly publish them based on their response. Maybe these choices are reasonable, and maybe they aren’t—you raised some really good points I think. But it just seems hard to evaluate in a vacuum.
LIC has a historical track record, and it is a bad one. People should have the opportunity to start something new. However, they shouldn’t be rated a top charity after receiving over a million dollars in funding and failing to achieve any positive legal outcomes.
Again, I don’t really see good evidence for this—what is the typical track record for legal campaigns? How much do they cost? How long do they take to work? These all would be important questions to answer before claiming cost-effectiveness or lack thereof. In this case, I could easily be persuaded to agree with you, but not for any of the reasons in your analysis — the fact that they spent some money some lawsuits and it didn’t work isn’t the only evidence I’d want to think about whether or not donations to them will be useful.
Did you ask ACE to review this before publishing? It seems like the kind of thing that would be worth getting feedback on before publishing. I didn’t look at this for more than a couple minutes, but I saw immediately that there might be some conceptual disagreements between you and ACE—for example, I noticed that in your first example, you assume in your example (I believe), that if LIC didn’t spend 200k on the lawsuit against Costco, they wouldn’t spend it on anything else. It’s unclear to me that this is the counterfactual, or how ACE is conceptualizing those funds. There might be reasoning behind their decisionmaking that would be useful to your critiques they could share.
I also felt like this felt pretty politically motivated. Not sure if that is your intention, but paragraphs like this:
ACE’s recommendations determine which animal charities receive millions of dollars in donations.[1] Thus far, we have reviewed 5 of ACE’s “Top 11 Animal Charities to Donate to in 2024” and only one of them (Shrimp Welfare Project) appears to be an effective charity for helping animals. ACE’s poor evaluation process leads to ineffective charities receiving recommendations, and many animals are suffering as a result.
Without any evidence feels pretty intense. ACE is kind of low hanging fruit to pick on in the EA space, so this read to me like more of that, without necessarily the evidence base to back it. Reading your report, I felt kind of like “oh, there are interesting assumptions here, would be interested to learn more”, and not “ACE is doing an extremely bad job.”
E.g. I think the questions that would be good to ask in a critique of ACE might be:
If ACE didn’t exist, how would the funds the direct be spent otherwise? Would that be better or worse for animals?
Is historical track record / cost-effectiveness the only lens on which to evaluate charities?
If the answer is yes, seems very hard to start new things!
I don’t know if the LIC legal case is this, but celebrating the potential impact of promising bets that didn’t pan out seems good to me.
I also think getting feedback on statements like this would be really helpful:
The correct formula for calculating cost-effectiveness is simply impact divided by cost. Rather than using this simple formula, ACE has elected to create a methodology that does not properly account for impact or cost.
I think ACE has wanted to do this at points in their history — my impression is just that it is incredibly difficult, so they’ve approached it from other angles instead. I also don’t think it’s clear to me that ACE’s goal is to report cost-effectiveness. I think clarifying this with them, and getting a sense of why they don’t do what you see as the simple approach would be useful for making this critique stronger. And, I don’t think people should make giving decisions based only on historic cost-effectiveness—just because an opportunity was impactful doesn’t mean the organization needs more funds to do that work, that it will scale, work in the future, etc.
I don’t disagree that ACE might be directing funds to ineffective charities! I don’t really think non-OpenPhil EA donors should give to farmed animal welfare, for example. But, I don’t think it is obvious to me that ACE going away means money going to more effective charities—I expect it would mostly be worse—people giving to animal charities with basically no vetting.
That being said, critique of critical organizations is great in my opinion, so appreciate you putting this out there!
Note that we now have raised $3,015 in pledges for November, and the marginal $25 donation will influence the allocation of around $234. In October, the average donation influenced $221.50, so if you’re excited to directly influence the allocation of funding between EA causes, this is still a great way to give right now! You can sign up here.
Thanks! Strongly agree with making it more democratic via some mechanism, and if it survives beyond the first 6 months, I plan on moving it to having some kind of elected oversight group or similar (mainly will figure out how to do that with input from the members). Interesting note on sortition—this seems plausibly like a good use for it. Thanks!
The first month of Equal Hands is complete!
Here are the results: 20 donors pledged to give $4,430 according to the collective preference of the pool.
This resulted in the following donations:
October 2024 AW GHD GCR EA Community Climate Change Total Pledge Breakdown 39.25% 26.20% 27.40% 3.75% 3.40% 100% Implied Donations $ 1,738.78 $ 1,160.66 $ 1,213.82 $ 166.13 $ 150.62 $ 4,430.00 Pseudo Counterfactual $ 2,711.00 $ 1,172.25 $ 354.75 $ 81.50 $ 110.50 $ 4,430.00 Implied Change -$972.23 -$11.59 +$859.07 +$84.63 +$40.12 All but 2 donors met their pledge, and $4,355 was given following the system. Backstopping funders covered the $75 gap left by the two donors.
Interestingly, the net effect (compared to the pseudo counterfactual of the money being distributed by each donor according purely to their preferences) of Equal Hands in October was roughly to move ~$900 from animal welfare to GCR areas. From the data, it looks like the primary cause of this was that animal welfare-motivated donors were most likely to give the largest amounts, but GCR donors were more likely to sign up (especially at the minimum, $25).
We’re running 5 more months of this trial, and you can sign up here.
Yeah, I think that’s basically what I was thinking (specifically, starting an insecticide charity, or similar project focused on implementing a WAW intervention)
I think it would be pretty hard for me to make that trade off in a workplace context (I think I’m still a deep sucker for impact and in any real version of this is X would be whatever the organization is indifferent towards and I’d donate it). If you forced me to in some hypothetical I’d guess X is quite low for many junior roles (<$10k), but higher for more mid/senior roles (>$50k?). But I think something like the following are true:
I’m currently not doing what I suspect would be the most impactful jobs for me to do, in part because what seems reasonable to pay for them (based on market rates, etc) strikes be as being at least $30k-$40k below what I would consider.
As recently as a few years ago, I probably would have considered them at that level.
My expenses haven’t changed in any meaningful way (outside inflation, etc).
I think the work I’m doing instead is almost certainly significantly less impactful.
I think this is bad, but compensation isn’t the only consideration on my mind.
I think generally past a certain point, having (or moving) money is strongly correlated having strategic influence within certain spaces in EA, so it seems pretty important.
This is obviously not necessarily correlated with having strategic skill
Nice—that’s good to know—I was under the impression that it was a good idea, but didn’t get much traction.
I’ve definitely heard speciesism used both ways, but I think it’s usually used without much reference to an exact view, but as a general “vibe” (which IMO makes it a not particularly useful word). But, I think people in the EA-side of the animal advocacy world tend to lean more toward the “it’s discriminatory to devalue animals purely because they aren’t a member of the human species” definition. I’d guess that most times its used, especially outside of EA, it’s something more like the “it’s discriminatory to not view all animals including humans as being of equal value” view but with a lot of fuzziness around it. So I’d guess it is somewhat context dependent on the speaker?
Sorry to just see this!
I agree that for many individuals, going vegan could be a good way to help animals! It’s not obvious to me that it is easier to do for most of those people than say, donating to a charity at a rate that roughly offsets the harm from it. I don’t really think the specific harm of “eating animals” is worse than the variety of other ways that we eat animals, so feel pretty neutral about veganism — it seems like one of many effective things one can do personally to help animals.
I basically don’t know if I believe we’ll find anything amazingly effective to do for farmed animals beyond cage free campaigns on this timeline, and those impact a pretty small portion of farmed animals.
I feel confused by this personally—I don’t think it makes sense that I’d have an obligation to bring positive lives into existence, and feel like there should be some symmetry here, but it doesn’t feel exactly the same. I also don’t think 35% is a small probability! It seems not unlikely to me that I have this kind of obligation.
I think EAs won it because they spent a lot of money on things that actually worked (e.g. cage-free campaigns) instead of wasting money on diet change advocacy that wasn’t very effective, etc. And I think it was good because it actually did something to help animals! I generally just think the EA side of the animal welfare space is more interested in evidence, and less in ideological purity. These both seem very good to me!
Not really, primarily because I don’t think the animal welfare world currently has the organizational competency to do any of them successfully at that scale, and not shoot itself in the foot while doing so, with the potential exception of the advance market commitments. I don’t think the existing groups have the organizational competency to handle the ~$200M they already receive well, and think the majority that money is already being spent in expectedly worse ways than giving to GiveWell top charities, even if the best animal stuff is incredibly cost-effective. I think that the movement could get there at some point. But if I imagine that much money going to any existing group to be spent in the next 2 years I think it would mostly be wasted.
I think many of these ideas seem feasible in the longrun, and are viable candidates for what to try, though I just generally think that farmed animal welfare is significantly less tractable than wild animal welfare or invertebrate welfare in the longrun, so would rather the funds went to scaling those fields instead of farmed animal welfare. Also, it is not obvious to me that lots of these ideas will beat out global health charities, though I think blue sky thinking is good.
Also just generally, most of those ideas are ones that don’t need to be implemented at scale? E.g. Healthier Hens doesn’t seem like it has been able to demonstrate that it is cost-effective to donors at a small scale. Why would scaling it up 1000x go better? It seems like if these ideas could absorb $100M, many could be tried now. The one that hasn’t been tried at that scale is advance market commitments, but I think the track record for alternative proteins doesn’t look great in general right now, and it isn’t obvious to me that R&D is the main barrier — see the margarine issues.
I also generally think lots of untried ideas look good on paper, but will probably not end up being effective if tried. That doesn’t mean we shouldn’t try them, but I think the bar has to be higher than “beats GiveWell in expectation from current evidence,” because the uncertainty is also a lot higher.
I think that if I were allocating this funding, there is a very low chance I’d choose to allocate any significant portion of it to farmed animal welfare, given that it isn’t nearly as neglected as other larger scale animal issues, and I don’t think there are good opportunities on the horizon at scales larger than OpenPhil’s animal welfare budget. If OpenPhil stopped funding animal welfare entirely, I’d likely want to see something like $50M going to farmed animal welfare, and almost entirely to corporate campaigns for shrimp as well as some cage-free clean up work, and maybe something in the near future on fish that no one has figured out yet.
If I had to guess at “the fastest way we could spend $100M on animals extremely effectively”, I’m think it will be something like putting some research into insecticide interventions and scaling them a lot, and definitely nothing implemented by existing farmed animal groups. If there was anything in the farmed animal space, it would be research, but again—I’m skeptical there are good opportunities beyond what OpenPhil can already fund.
I feel pretty disappointed by a lot of the above—I spent several years professionally working on corporate campaigns, and am as animal friendly as they come, but I’ve just heavily decreased my confidence in the actual scale of tractable opportunities to improve farmed animal welfare as a whole over the last few years — in large part because it seems like very little has worked despite lots of money being poured into the space.
There are 7 days left to sign up for the first month of this experiment!
As of right now, the marginal $25 influences the allocation of about $253 in expectation.
Nice—yeah, it wouldn’t surprise me if that period is slower than the last ~5 years, as a lot of the capital that has gone into the space seems like it has been spent, and it doesn’t seem like recent capital inflows have been as high. My 4-7x guess is based on a crude estimate done by Sagar Shah of how much production capacity can be bought with the capital that companies have available to do it, with a delay baked in for construction time.
What do you mean by “invest” here?
I meant more literally, put $100M in an investment account to save for good future animal opportunities vs spending on the best global health interventions today. I’m not certain it’s actually a 50⁄50 item, but was trying to find a mid point.
Maybe there are better global health interventions that can absorb $100M over time than GiveWell recommendations, though.)
I don’t really know enough about global health work to say—but I’d guess there are some novel medical things seem plausibly able to:
Appear over the next few decades
Require a lot of cash to scale up
Could be really cost-effective
Yes correct—just the Insects as Food and Feed industry. Though note these estimates were from 2020 - my best guess is that there are at least 4-7x as many insects farmed by the industry today (mainly because it’s going through a lot of industrialization / scale up, and a bunch of new major factories have opened in the last few years).
I don’t really have a strong view about LIC—as I’ve mentioned elsewhere in the comments, I’m skeptical in general that very EA donors should give to farmed vertebrate welfare issues in the near future. But I don’t find this level of evidence particularly compelling on its own. I think I feel confused about the example you’re giving because it isn’t about hypothetical cost-effectiveness, it’s about historic cost-effectiveness, where what matters are the counterfactuals.
I broadly think the critique is interesting, and again, seems like probably an issue with the methodology, but on its own doesn’t seem like reason to think that ACE isn’t identifying good donation opportunities, because things besides cost-effectiveness also matter here.