To add, I also have issues with ACE’s methodology & transparency. I don’t have a background in stats or anything, but when I tried to independently verify ACE’s evaluation of charity efficacy, I found several figures that seem (to me) not well-sourced. e.g., the ‘source’ column is mostly empty for ACE’s AWO spreadsheet: https://docs.google.com/spreadsheets/d/1CAcmf4dXPk9tNWit25C7A7GJqEGNtnN-unXKZoyzvx4/edit
As I get more familiar with EA and will soon enter the workforce, I am considering how to decide on impactful future donations. I have been a vegetarian for 6 years and am greatly preoccupied by animal farming. Although the abolitionist perspective is attractive, it does not seem achievable to me, so I wish to orient donations towards animal welfare. This post makes me concerned about the lack of research and fake promises.
I agree with the other, secondary Aidan’s comment (I am the original and true Aidan).
A common journey with EA is going from (1) not thinking we can know whether charity is helping or that it makes a difference which charity you choose to (2) learning about EA and being thrilled to discover that we can know and some charities are excellent to (3) getting into the weeds of the empirical and ethical uncertainties around the first, 2nd and nth order effects of a certain cause area and becoming disillusioned and pivoting to another cause to (4) realising that every cause is fraught with uncertainty and starting to feel a bit jaded about charity to (5) accepting that the world is super uncertain but that the expected impact of the best bets we have is promising enough to motivate ongoing work on them (especially if we double down on generating more primary evidence and doing good M&E to make our uncertainty go down with time)
I hope we don’t lose you at the 4th step ❤️
(This is not to say that all causes are equally uncertain, or that it’s guaranteed that the best bets in any particular cause will be compelling enough for you)
I really like this description and it resonates with me. +1 to “I hope we don’t lose you at the 4th step” :)
It’s so desperately sad to have uncertainty about extremely important things, and yet we still need to choose how to act. It feels very productive to discuss where to allocate marginal resources between “reduce our uncertainty further” and “take the best step according to our current evidence base”, and I’m glad to see it being discussed.
Congrats/good luck on soon entering the workforce!
I think OP is a valuable contribution to an ongoing discussion, but I’d encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸’s comment above.
“There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons” — I agree, with the caveat that sometimes what looks like (A) converging opinions of a number of thoughtful actors is actually (B) a chain of deference where one actor has thought about it, and a second actor defers to them, and a third defers to the first two etc, with the nuance and uncertainty getting lost a bit more at every step in the chain. It can be hard to tell A and B apart from the outside
Numerous EA-adjacent orgs arriving at the same conclusion about some issue may also be the result of re-circulating the same people. After one year of observing EA online, my impression is that EA is not that large or diverse a group of people in the grand scheme of things. Many people seem to be pretty tightly interconnected together, even to the point of being family with each other!
I actually think EA does a pretty good job of avoiding group-think relative to its homogeneity (see posts like OPs), but given the social dynamics of a smaller and tightly interconnected movement, it is important to constantly reinforce truth-seeking behaviors, including by direct questioning of established orthodoxy.
Maybe my characterization of EA is wrong though. I’m not someone who would know.
I think what really bothers me about OPs post, is precisely the possibility that my donations are actually worsening animal welfare. That stings. I think its important for recommenders of interventions to carefully consider how they are going to communicate about such things. I would want them to break out not only their general uncertainty about the effectiveness of the intervention, but the specific uncertainty that it actually causes harm. For those of us who are concerned a lot about harm reduction, seeing that as its own line item would be helpful.
It’s very fair to be concerned that your donations might be doing harm—I can relate! A bittersweet consolation might be that this is a risk that applies to basically every charitable intervention, including highly evidence-based global health and development interventions. Something that increases a woman’s income or autonomy can put her at greater risk of domestic violence for example. Changing someone’s life can have impacts on the local and macro-economy, on those impacted by their diet and consumption patterns etc. -- the world is complicated and the risk of doing harm instead of good is always there.
This isn’t to say we shouldn’t work really hard to understand and prevent such harm! Just that “no chance of harm” is not a realistic bar to aim for
Adding to what Other Aidan said, I think it’s a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from “very slightly helpful” to “very slightly harmful” as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is −10 units, the difference between A and B matters much more than between B and C.
I agree with you two. I don’t have any delusions about avoiding all risk of causing harm or that harm avoidance is straightforwardly more important than providing benefits. I guess what I am saying is that risk of causing harm is distinct from risk this is ineffective, and it would be nice to see these broken out (as someone who works in data analytics and engineering, I realize the real world is not so simple). It seems to me that actively causing harm is a bit of a different thing than just being ineffective, and you would ideally reason about it specifically, rather than bundling it all together into one big effectiveness metric.
Especially since, while I don’t think we should try to avoid all harm, people may have different moral weights about causing harm. For some people, they may be much more indifferent about causing harm relative to providing some benefit, whereas others may have a stronger bias towards “first, do no harm.” Given that the tradeoff between these is something each individual must determine, it is better to separate it out in your model and allow people to discount the effectiveness according to their own priorities.
Yes, this!
To add, I also have issues with ACE’s methodology & transparency. I don’t have a background in stats or anything, but when I tried to independently verify ACE’s evaluation of charity efficacy, I found several figures that seem (to me) not well-sourced. e.g., the ‘source’ column is mostly empty for ACE’s AWO spreadsheet: https://docs.google.com/spreadsheets/d/1CAcmf4dXPk9tNWit25C7A7GJqEGNtnN-unXKZoyzvx4/edit
Compare with GiveWell’s analysis: https://www.givewell.org/international/technical/programs/seasonal-malaria-chemoprevention#Sources
I’m still allocating my donations based on ACE’s recommendations, but I’m really worried about how effective I’m being.
Thank you for your insights @matthes @Citrit !
As I get more familiar with EA and will soon enter the workforce, I am considering how to decide on impactful future donations. I have been a vegetarian for 6 years and am greatly preoccupied by animal farming. Although the abolitionist perspective is attractive, it does not seem achievable to me, so I wish to orient donations towards animal welfare. This post makes me concerned about the lack of research and fake promises.
I agree with the other, secondary Aidan’s comment (I am the original and true Aidan).
A common journey with EA is going from (1) not thinking we can know whether charity is helping or that it makes a difference which charity you choose to (2) learning about EA and being thrilled to discover that we can know and some charities are excellent to (3) getting into the weeds of the empirical and ethical uncertainties around the first, 2nd and nth order effects of a certain cause area and becoming disillusioned and pivoting to another cause to (4) realising that every cause is fraught with uncertainty and starting to feel a bit jaded about charity to (5) accepting that the world is super uncertain but that the expected impact of the best bets we have is promising enough to motivate ongoing work on them (especially if we double down on generating more primary evidence and doing good M&E to make our uncertainty go down with time)
I hope we don’t lose you at the 4th step ❤️
(This is not to say that all causes are equally uncertain, or that it’s guaranteed that the best bets in any particular cause will be compelling enough for you)
I really like this description and it resonates with me. +1 to “I hope we don’t lose you at the 4th step” :)
It’s so desperately sad to have uncertainty about extremely important things, and yet we still need to choose how to act. It feels very productive to discuss where to allocate marginal resources between “reduce our uncertainty further” and “take the best step according to our current evidence base”, and I’m glad to see it being discussed.
Congrats/good luck on soon entering the workforce!
I think OP is a valuable contribution to an ongoing discussion, but I’d encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸’s comment above.
“There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons” — I agree, with the caveat that sometimes what looks like (A) converging opinions of a number of thoughtful actors is actually (B) a chain of deference where one actor has thought about it, and a second actor defers to them, and a third defers to the first two etc, with the nuance and uncertainty getting lost a bit more at every step in the chain. It can be hard to tell A and B apart from the outside
Numerous EA-adjacent orgs arriving at the same conclusion about some issue may also be the result of re-circulating the same people. After one year of observing EA online, my impression is that EA is not that large or diverse a group of people in the grand scheme of things. Many people seem to be pretty tightly interconnected together, even to the point of being family with each other!
I actually think EA does a pretty good job of avoiding group-think relative to its homogeneity (see posts like OPs), but given the social dynamics of a smaller and tightly interconnected movement, it is important to constantly reinforce truth-seeking behaviors, including by direct questioning of established orthodoxy.
Maybe my characterization of EA is wrong though. I’m not someone who would know.
I think what really bothers me about OPs post, is precisely the possibility that my donations are actually worsening animal welfare. That stings. I think its important for recommenders of interventions to carefully consider how they are going to communicate about such things. I would want them to break out not only their general uncertainty about the effectiveness of the intervention, but the specific uncertainty that it actually causes harm. For those of us who are concerned a lot about harm reduction, seeing that as its own line item would be helpful.
It’s very fair to be concerned that your donations might be doing harm—I can relate! A bittersweet consolation might be that this is a risk that applies to basically every charitable intervention, including highly evidence-based global health and development interventions. Something that increases a woman’s income or autonomy can put her at greater risk of domestic violence for example. Changing someone’s life can have impacts on the local and macro-economy, on those impacted by their diet and consumption patterns etc. -- the world is complicated and the risk of doing harm instead of good is always there.
This isn’t to say we shouldn’t work really hard to understand and prevent such harm! Just that “no chance of harm” is not a realistic bar to aim for
Adding to what Other Aidan said, I think it’s a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from “very slightly helpful” to “very slightly harmful” as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is −10 units, the difference between A and B matters much more than between B and C.
I agree with you two. I don’t have any delusions about avoiding all risk of causing harm or that harm avoidance is straightforwardly more important than providing benefits. I guess what I am saying is that risk of causing harm is distinct from risk this is ineffective, and it would be nice to see these broken out (as someone who works in data analytics and engineering, I realize the real world is not so simple). It seems to me that actively causing harm is a bit of a different thing than just being ineffective, and you would ideally reason about it specifically, rather than bundling it all together into one big effectiveness metric.
Especially since, while I don’t think we should try to avoid all harm, people may have different moral weights about causing harm. For some people, they may be much more indifferent about causing harm relative to providing some benefit, whereas others may have a stronger bias towards “first, do no harm.” Given that the tradeoff between these is something each individual must determine, it is better to separate it out in your model and allow people to discount the effectiveness according to their own priorities.