I see myself as a generalist quantitative researcher.
Vasco Grilošø
Karma: 7,057
Thanks, Karolina! Great updates.
Great analysis, Isaac! I worry the Animal Welfare Fund (AWF) has similar problems (see below), but they are way less transparent than ACE about their evaluations, and therefore much less scrutable. Instead of mostly deferring to AWF, I would rather have donors look over ACEās evaluations, discuss their findings with others, and eventually publish them online, even if they spend much less time on these activities than you did.
AWF only runs cost-effectiveness analysis (CEAs) for a minority of applications. According to a comment by Karolina Sarek, AWFās chair, on June 28 (this year):
In the past, we tended to do CEAs more often if: a) The project is relatively well-suited to a back-of-the-envelope calculation b) A back-of-the-envelope calculation seems decision-relevant. At that time, a) and b) seem true in a minority of cases, maybe ~10%-20% of applications depending on the round, to give some rough sense. However, note that there tends to be some difference between projects in areas or by groups we have already evaluated versus projects/āgroups/āareas that are newer to us. Iād say newer projects/āgroups/āareas are more likely to receive a back-of-the-envelope style estimate.
Comparisons across grants also seem to be lacking. From Giving What We Canās (GWWCās) evaluation of AWF in November 2023 (emphasis mine):
Fourth, we saw some references to the numbers of animals that could be affected if an intervention went well, but we didnāt see any attempt at back-of-the-envelope calculations to get a rough sense of the cost-effectiveness of a grant, nor any direct comparison across grants to calibrate scoring. We appreciate it wonāt be possible to come up with useful quantitative estimates and comparisons in all or even most cases, especially given the limited time fund managers have to review applications, but we think there were cases among the grants we reviewed where this was possible (both quantifying and comparing to a benchmark) ā including one case in which the applicant provided a cost-effectiveness analysis themselves, but this wasnāt then considered by the PI in their main reasoning for the grant.
GWWC looked into 10 applications:
Of the 10 grant investigation reports we reviewed, three were provided by the AWF upon our general request for representative grants; two were selected by us from their grants database; two were selected by the AWF after we provided specifications; and three were selected by the AWF based on our request for grant applications by organisations that applied to both the AWF and ACEās MG.
Karolina also said on June 28 that AWF has improved their methodology since GWWCās evaluation:
However, since then, weāve started conducting BOTEC CEA more frequently and using benchmarking in more of our grant evaluations. For example, we sometimes use this BOTEC template and compare the outcomes to cage-free corporate campaigns (modified for our purposes from a BOTEC that accompanied RPās Welfare Range Estimates).
I do not doubt AWF has taken the above steps, but I have no way to check it. I think donating to ACE over AWF is a good way of incentivising transparency, which ultimately can lead to more impact.
Thanks, Alene!
I think total animal welfare is currently decreasing because factory-farming is increasing in low and middle income countries due to population and economic growth. However, I believe animal welfare per person is already increasing in countries where welfare reforms have been successful, and I expect these to eventually spread globally. Air pollution and greenhouse gas emissions also dramatically increased since the industrial revolution, started by going down in high income countries, and I expect they will eventually go down globally.
In any case, even if positive factory-farming was inevitable (some time in the future), getting there faster would still be super valuable!
Thanks, Aaron.
Another point to emphasise thoughāitās my sense that the intervention should be modelled as electrical stunning replaces air asphyxiation, rather than (perfectly implemented) ice slurry. Ice slurry slaughter is just a very difficult thing to do correctly in practice (and Iāve not seen it happen) - as even if at some point the shrimps are submerged in ice for a short period of time, itās often not long enough to kill them (~30seconds).
I accounted for badly implemented ice slurry slaughter. I assumed:
All of the shrimps helped transition to electrical stunning, 62.5 % (= 1 ā 0.375) from air asphyxiation, and 37.5 % from ice slurry (= 0.75*0.5). I got these fractions assuming 75 % of the targeted producers use some form of ice slurry, half of those implement it properly, and the other half improperly to the point of it being practically equivalent to air asphyxiation. I made these assumptions having in mind Aaronās comment at the end of this section.
In any case, based on my assumptions, it does not matter whether HSI is harmful for the 37.5 % of the affected shrimp which go from well implemented ice slurry to electrical stunning slaughter. The overall cost-effectiveness is dominated by making 62.5 % of the affected shrimp go from air asphyxiation to electrical stunning slaughter. I estimate 97.3 % (= 0.625*0.0447/ā0.0287) of the benefits come from helping shrimp slaughtered via air asphyxiation, and that the increase in welfare for these is 48.6 % (= 1 ā 4.85/ā9.44), whereas the number of shrimp would very hardly increase by that. So the question of whether the number of shrimp increases is only relevant if a very small fraction of the helped shrimp is slaughtered via air asphyxiation (again, conditional on my assumptions).
Thanks for the comment, Tristan.
I agree factory-farmed animals having positive lives is not sufficient to justify continuing factory-farming. If one is confident that i) wild animals have positive lives, ii) the effects of factory-farming on wild animals are much larger than those on farmed animals, and iii) factory-farming decreases the number of wild animals (as animal products usually require more land than plant-based ones), then ending factory-farming would be good to increase animal welfare when accounting for both farmed and wild animals. However, I am not arguing for reaching positive factory-farming, and then maintaining it forever. I am arguing for having the ultimate goal of increasing animal welfare, which I think is robustly good, unlike ending factory-farming or maintaining positive factory-farming forever.
I also think that focussing on increasing the welfare of factory-farmed animals (instead of ending factory-farming) is more conducive to making people care about the welfare of wild animals. People justify ending factory-farming as a way of perserving nature, but I believe this may well backfire because wild animals may have negative lives. There is huge uncertainty about whether wild animals have positive/ānegative lives, so I would say it is more prudent to have the robustly good ultimate goal of increasing welfare.
Thanks for clarifying, Joey!
I think it depends quite a bit on the quality of the CEA. I would take a sub-5-hour WFM as more useful than a sub-5-hour CEA every time. At 50 hours, I think it becomes a lot less clear. CEAs are much more error-prone and more punishing of those errors compared to WFMs, thus the risk of weaker CEAs.
I agree the value of CEAs relative to a WFM increases with time invested.
I also think the assessment of GW as CEA-focused is a bit misleading. They have four criteria, two of which they do not explicitly model in their CEA, and many blog posts express their skepticism about taking CEAs literally (my favorite of these, though old).
Elie Hassenfeld (GiveWellās co-founder and CEO) mentioned on the Clearer Thinking podcast that (emphasis mine):
GiveWell cost- effectiveness estimates are not the only input into our decisions to fund malaria programs and deworming programs, there are some other factors, but theyāre certainly 80% plus of the case.
Isabel Arjmand (GiveWellās special projects officer at the time) also said (Isabelās emphasis):
The numerical cost-effectiveness estimate in the spreadsheet is nearly always the most important factor in our recommendations, but not the only factor. That is, we donāt solely rely on our spreadsheet-based analysis of cost-effectiveness when making grants.
Interesting question, David!
I think it is very unclear whether wild animals have positive/ānegative lives, so I guess it is fine to neglect the effects on wild animals of interventions aiming to improve the welfare of farmed animals or humans. I have posted about these effects, and I believe their discussion can still be useful as a way of raising awareness for wild animal welfare.
In addition, as with the meat-eater problem, I suspect the effects on wild animals are mostly a distraction for cause prioritisation. If one is confident the effects on wild animals are positive/ānegative, and that their magnitude is significant, then I would expect interventions explicitly aiming to improve the welfare of wild animals to be more cost-effective than those targetting farmed animals or humans.
To clarify for certain readers, I think that the right of the woman to family planning comes first and shouldnāt be restricted. I think that foregone positive lives only has implications for prioritising among our positive obligations, not negative rights or liberties. Iām sure you agree.
My views are pretty close to Ariel Simnegarās.
Thanks, Joey and Ben.
Number of meaningfully different heuristics considered (the criteria in the columns of a WFM)
Number of meaningfully different solutions considered (the options to be evaluated in the rows of a WFM)
For cost-effectiveness analyses (CEAs) and a WFM covering the same variables and options, would the CEAs be preferable? I think so. For example, I like that GiveWell uses cost-effectiveness analyses of the most promising countries instead of a WFM.
You are welcome!
we should still be considering flow-through effects in questions of cause prioritization and which GHD interventions to support. I think there are also reasonable edge cases where we may be able to influence GHD interventions to have better positive flow-through effects, if the donor is not onboard with AW.
Agreed. By āI worry the meat-eater problem is mostly a distractionā, I meant a distraction for cause prioritisation. My sense is that people (like me) who consider the meat-eater problem to be a relevant consideration for prioritising within global health and development value 1 unit welfare in humans sufficiently similarly to 1 unit of welfare in animals to conclude that the best animal welfare interventions are much more cost-effective than the best human welfare interventions.
Thanks for the good context, Jeff.
I donāt think that would work very well
I think a good part of the appeal is that it is low cost strategy (relative to one where lots of context is provided).
Our main funders have a lot of context on our work, and so our grant applications are missing a lot of information that a typical Forum reader would need. This includes basic stuff like ā what problem are you trying to solve?ā
Note the target audience will be more familiar with the work than a random EA Forum reader. For example, if you were raising funds for the Nucleic Acid Observatory (NAO), your target audience would be more familiar with bio and more likely to have read other posts about NAO.
There is a lot of information you can share in a private grant request that you canāt make public.
Fair! Some information would have to be removed.
Thanks for the comment, Nithin.
This may negate any decrease in human welfare, but I havenāt seen a BOTEC of this that models the income increase on the meat eating problem.
As suggested by the graph below, the increased income of the helped families will tend to increase their consumption of animals, which is harmful if animals have negative lives.
However, since I think family planning interventions decrease human population, I believe they decrease the consumption of animals. Yet, I am not sure this is good because farmed animalsā lives may become net positive in the next few decades, and the children who would be born from unwanted pregancies would live longer than that.
In any case, I worry the meat-eater problem is mostly a distraction. If one values 1 unit of welfare in animals as much as 1 unit of welfare in humans, and does not think Rethink Prioritiesā welfare ranges are wildly off, the best animal welfare interventions will be much more cost-effective than the best global health and development interventions. I estimate the cost-effectiveness of Shrimp Welfare Projectās (SWPās) Humane Slaughter Initiative (HSI) is 43.4 k times that of GiveWellās top charities.
it does seem like some cows probably have net-positive lives right now
Agreed. I guess farmed animals have positive lives under the conditions required by the Naturland standard.
Thanks for the post, Jeff. Sharing lots of relevant information would take time, and I think applying to EA Funds and Open Philanthropy would raise funds more cost-effectively. However, orgs can simply share their applications to larger funders on EA Forum, and then add info about what they would do with marginal funding on top of funds received from the larger funders.
Thanks for the comment. Joey Savoie, the director of strategy of Charity Entrepreneurship, which incubated Lafiya Nigeria, said:
I am far less convinced that life saving interventions are net population creating than I am that family planning decreases it.
It looks like life saving interventions decrease fertility, but still increase population. So the above suggests family planning interventions do decrease population.
The section āHow Many People Have Positive Wellbeing?ā of Chapter 9 of What We Owe the Future mentions a few data points about neutral life satisfaction:
The relative nature of the scale means that it is difficult to interpret where the neutral point should be, and unfortunately, there have been only two small studies directly addressing this question. Respondents from Ghana and Kenya put the neutral point at 0.6, while one British study places it between 1 and 2.
I think Nigeria is more like Ghana and Kenya than the United Kingdom, so people in Nigeria may put the neutral point at around 0.6. From the 2024 World Happiness Report, Nigeria had a mean life satisfaction from 2021 to 2023 of 4.88, which is significantly higher than 0.6. I assume unwanted pregnancies will be more frequent in families with life satisfaction below the mean, but 4.88 is 8.13 times (= 4.88/ā0.6) as large as 0.6, so I expect a random child that would have been born from a prevented unwanted pregnancy to have a positive life.
My low confidence best guess is that Lafiya Nigeria decreases human welfare after accounting for the effect above. Difference assumptions may lead to different conclusions, but I believe one should at least discuss the potential loss of welfare of the children whose lives are prevented.
I would be curious to know your thoughts, @Klau Chmielowska. Thanks anyway for your hardwork.
Thanks for sharing. Do you think children born from unwanted pregnancies have positive lives? If so, would the family planning intervention still be beneficial accounting for the welfare loss of the children who would have been born from the prevented unwanted pregnancies? This seems like a crucial consideration.
Thanks, Elliot.
Many animal advocates frame the goal of the movement as āending factory farmingā.
I see why itās a tempting message, both to hold onto internally, and when pitching to people new to the movement.
Yet, I think the reality is that we might never get there.
I think the framing therefore leads to the following problems:
Unrealistic hope leads to disillusionment and burnout.
You should count counterfactual wins, not the absolute numbers.
A lack of strategic clarity when developing a theory of change.
Leads to a poor allocation of resources.
There is another point which makes me especially in favour of focussing on reducing suffering, and also increasing happiness. Ending factory-farming only increases animal welfare if factory-farmed animals continue to have negative lives forever, whereas I would say they may become positive in the next few decades at least in some animal-friendly countries.
Thanks for the great context, Aaron!
Is there any chance HSI may increase the number of shrimp? I guess it would tend to increase costs, and therefore decrease the number of shrimp. I ask because I estimate that moving from ice slurry to electrical stunning only increases welfare by 4.34 % (= 1 ā 4.85/ā5.07). In this case, since I think farmed shrimp have negative lives (for any slaughter method), an increase of more than 4.34 % in the number of shrimp would make HSI harmful.
Thanks for another relevant question too! I do not think that alone would make dairy production net negative:
According to CIWF, āDairy cows must give birth to one calf per year in order to continue producing milkā.
From Animal Australia, āboth mother and calf can often be heard calling out for each other for hours [after they are separated]ā.
Assuming disabling pain of 3 h/āyear for each the mother and child based on the above, one gets 6 h/āyear (= 2*3) of disabling pain. For my intensity of disabling pain, that corresponds to a loss of 0.00684 AQALY/āyear (= 6/ā24/ā365.25*10).
The above is quite small in comparison with the magnitude of the values I got for chickens and shrimp. So, in the absence of longer term effects from the separation, I do not think it would bring dairy production from positive to negative.
Hi Saulius,
What are your current best guesses for the expected chicken-years improved per $ for broiler welfare and cage-free campaigns funded in 2024? Open Philanthropy thinks āthe marginal FAW [farmed animal welfare] funding opportunity is ~1/ā5th as cost-effective as the average from Sauliusā analysis [this post]ā, which suggests broiler welfare and cage-free campaigns improve 3.00 (= 15*1/ā5) and 10.8 chicken-year/ā$ (= 54*1/ā5).