I think it is quite clear that a lot of your research isn’t at the bar of those other organizations (though I think for the reasons Joey mentioned, that definitely can be okay). For example, I think in this report, collapsing 30 million species with diverse life histories into a single “Wild bug” and then taking what appear to be completely uncalibrated guesses at their life conditions, then using that to compare to other species is just well below the quality standards of other organizations in the space, even if it is a useful way to get a quick sense of things.
mildlyanonymous
Karma: 179
I think being hostile was probably slightly too strong, though I will note that the original comment still has positive upvotes / many agree votes, but no other people defending this position in the comments, which is concerning, and mostly CE staff and incubatees responding
I gave a few examples here, but as I mentioned in response to Aidan, I don’t think that comment makes much sense as an overall defense.
My goal here is not to provide this to the EA Forum, but to caution donors about doing further due diligence. But I mentioned a few examples of more egregious research failures in another comment.
I’ll also add that the original comment still has positive comment karma and many agree votes, and that many of the disagree votes seem to be AIM staff and incubatees, and not necessarily others in animal welfare research. I think that at a minimum, that should raise flags for many people to take these concerns seriously.
It looks like the report has been taken down, but I think the degree to which you pushed dissolved water oxygenation for fish welfare before launching Fish Welfare Initiative is an especially strong example of this. At the time I heard skepticism from many experts. You can see a reference to that report in this post. This report is another example of something that I think would not have passed any kind of rigorous external review.
I think this is overly simplifying of something a lot more complex, and I’m surprised it’s a justification you use for this. Of course on some level what you’re saying is correct in many cases. But imagine you recommend a global health charity to be launched. GiveWell says “you’re misinterpreting some critical evidence, and this isn’t as impactful as you think”. Charities on the ground say “this will impact our existing work, so try doing it this other way”. You launch the intervention anyway. The founders immediately get the same feedback, including from trying the intervention, then pivot to coordinating more and aligning with external experts.
This seems much more analogous to what happens in the animal space, and it seems absolutely like a good indicator that people were skeptical. Charities aren’t for-profits, who exist in a vacuum of their own profitability. They are part of a broader ecosystem.
I definitely agree that organizations should pivot as they learn about how an intervention works in practice. I think the errors I refer to are more things of the type: a cursory glance from an animal welfare scientist could have told you your research was missing key considerations, and the charity would have not wasted time on the recommended intervention. These seem cheap to prevent and preventable issues.
Thanks for sharing this! It differs from the narrative I’ve heard elsewhere is critical ways, but I don’t really know much about this situation, and just appreciate the transparency.
It also concerns me that I’ve seen 5 instances of this post being disagree voted and strong downvoted, then a CE staff member commenting right after. I think those are obviously things people have a right to do if it is CE staff downvoting and disagreeing, but it means this post, outside of CE staff, might have fairly strong agreement from many people, which seems like an important note, given that it is still very positive karma, and without those votes might be positive agreement on balance.
I agree that detailed evaluations would be better than my narrative impressions. My main point is to warn people to do way more due diligence on CE. Even if the reputation is undeserved, the organization has a negative reputation among many in animal advocacy, especially in research and grant evaluation, and that is worth looking into.
I don’t really have strong views on how to allocate funds in the animal space, but I doubt it is through funding circles, which usually seem worse for charities even if they are better for donors and the space overall (e.g. the degree of dislike that charities have for the existing Farmed Animal Funders circle seems like an indicator of something important).
It would be helpful if you engaged with the plagiarism claims, because it is concerning that CE is running researcher training programs while failing to handle that well. I agree with the rest of what you say here as being tricky, but think that it is pretty bad that you publish the low confidence research publicly, and it’s led to confusion in the animal space.
+ 2.5 - I think if your ordering is significantly different, it’s probably fairly different than most people in the space, so that’s somewhat surprising/an indicator that lots of feedback isn’t reaching you all.
To be clear, I am certain that CE staff have not been invited to events in the animal welfare space due to impressions of your organization being unwilling to be cooperative.
My main view is that animal donors should seriously engage in a vetting process prior to taking large amounts of guidance on donations from CE / shouldn’t update on your research in meaningful ways. I still think CE is probably the best bet in the animal space for future new very high impact organizations in the space as well though, so it’s a tricky balance to critique CE. I’d bet that a fair number of the best giving opportunities in the animal space in 5 years will have come out of CE, but that it’ll also have come with a large amount of generally avoidable wastes of funding and talent.
I’m mainly trying to convey what seemed to be a sentiment among many who worked in research in animal advocacy in response to seeing these ideas, though I agree with your second point.
As an example of this, I think that people thought Animal Ask or Healthier Hens both failed account for why the animal space had consolidated work on a few specific asks over the last few years (because corporations weren’t sure how to prioritize across many asks, and focusing on just one at a time helped get their attention to be more focused), and this was feedback conveyed to CE ahead of time but mostly ignored, and then became a route to failure for their early work.
(posting anonymously due to working closely with some CE groups)
I like CE a lot, and think that some of their charities are great. I donate to multiple CE charities (including one animal welfare charity). I appreciate their ambition and think that what they do is very difficult, and they’ve had a lot of success in the face of that difficulty.
However, I’m very concerned about their work in the animal welfare space, and wanted to flag some of these concerns if they are considering expanding in this space. I don’t think donors should take much guidance from them, compared to OpenPhil or the EA Animal Welfare Fund, and I would personally wager that CE leading giving in the animal space would be net-negative for the space compared to the status quo (which, to be fair, is very bad already). These comments are pretty light on a serious topic, but I want to flag them to donors considering joining this funding circle. These are one person’s impressions, and I’d encourage people considering joining these projects look into this more.
Concerns:
1. CE’s research on animal welfare is extremely low qualityThere is a widely held view in the animal research community that CE’s reports on animal welfare consistently contain serious factual errors, and their research is broadly not trusted by others in the space. My personal experience with this was reading a report I was an expert on, noticing immediately that it had multiple major errors, sharing that feedback, and having it ignored due to their internal time capping practices.
Another animal advocacy research organization supposedly found CE plagiarizing their work extensively including in published reports, and CE failed to address this.
2. CE’s charities working on animal welfare have mostly not been very good, and listening to external feedback prior to launching them would have told them this would happen.
Here are some very light evaluations of CE’s animal charities:Current CE incubated animal charities
Shrimp Welfare Project: Very promising, doesn’t do CE’s original proposed idea anymore, rough impression is that more feedback ahead of time would have told them the original idea was bad.
Animal Advocacy Careers: Okay, but not a super promising/scalable intervention, still does CE’s original proposed idea
Fish Welfare Initiative: Hasn’t worked very well, and seemed like it wouldn’t in advance, doesn’t do CE’s original proposed idea anymore, more feedback ahead of time would have told them not to do the original idea
Animal Ask: Was a bad idea prior to launching, hasn’t had much impact, people in the space were skeptical ahead of launch
Healthier Hens: Was a bad idea prior to launching, hasn’t had much impact, people in the space were skeptical ahead of launch
Animal Policy International: Was a bad idea prior to launching, too short a time has elapsed to assess
Incubating charities is difficult, and CE definitely shouldn’t expect 100% to work out. I think 1.25 charity out of 6 being good is still a quite solid success rate. But, most of these charities seemed like bad ideas to many other people in the space prior to launching, CE was given that feedback, and seemed to fail to act on it. This is also the case with their most recent batch of charities that haven’t yet launched. CE is typically much more confident in their own research than external feedback, which seems bad given concern 1.
3. CE is fairly hostile to external feedback on their animal welfare workCE has a fairly strong reputation of being hostile / non-collaborative in the animal welfare space. While their charities and founders tend to be very open to feedback and willing to work with others, CE itself has consistently been non-collaborative to other groups in the space to the degree that their staff are sometimes not invited to coordination events or meetings.
Using an anonymous account because this is sensitive. The below is critical, but I love GWWC, and personally agree with your recommendation of the EA AWF as the best opportunity for most donors in the AW space.
I’m quite surprised you chose to include The Humane League specifically and not other ACE Top Charities, and based on the evaluation, it sounds like this is largely based on RP and Founders Pledge work from 2018/2019, and not more recent evaluations, as well as a referral from Open Phil.
A few comments on why this seems like a bad decision:
My impression is that basically everyone within the animal space who has read them thinks those cost-effectiveness analyses are no longer accurate, and reflect an earlier period of success. Most people also seem to think corporate campaigns are no longer nearly as effective as they were historically (even staff within The Humane League).
Open Phil said just last week that they think marginal opportunities in the AW space right now are 1/5th as cost-effective as RP’s estimate (a claim I don’t buy fully, but seems like evidence against using this for your recommendation)
A cursory glance at the trackers for corporate commitments shows that:
Cage-free commitments secured annually have been massively declining (from a peak of around 500 per year in 2018/2019 when the analyses you used are from, to under 150 in 2023 (so far, though seems on track to not get to 150).
The vast majority of broiler commitments were secured prior to 2020. In 2023 only 6 (!) have been secured, all from fairly tiny companies.
It looks like the majority of recent commitments are from smaller companies (e.g. looks like the majority of “household names” in many countries committed years ago or have never committed), so my using “companies” as a metric of success is likely distortingly positive, so things could be even worse.
Most cage-free work now is focused on enforcement of past commitments, not new commitments.
This likely means the situation is even worse, as while dollars spent today might increase the number of animals helped, it likely means the estimates you used are overly optimistic (if they didn’t account for future spending needs to secure high levels of enforcement).
People who work in the space generally seem to view the broiler ask as bad / unlikely to be realized at any scale.
I haven’t really looked at this in detail, but I’m just pretty surprised you decided to recommend THL (and that THL is still the default giving option still in the animal space) without even checking the status of those analyses today, where I think they pretty obviously fail a basic sense check as still being accurate.
Obviously the historic figures no longer being accurate doesn’t necessarily mean THL isn’t the best option today, but it seems strange to not even glance at the current state of evidence prior to singling out a single charity when rejecting ACE’s evaluations. I personally suspect THL is no longer the best opportunity for marginal donations in the space by a somewhat wide margin, and would guess that most people who work in the space also don’t think this.
I’ll say something I said to Joey in this thread early—I expect that the best animal charities in the future will come out of AIM, but it will come with a lot of avoidable waste of funds and talent due to the things related to my concerns. I think AIM focusing on their skills at incubating charities, and less on what I believe are weaknesses or threats (coordinating donors and research), would be much better for the space.