I don’t like that this “converting to lives” thing is being done on this kind of post and seemingly nowhere else?
Like, if we applied it to the wytham abbey purchase (I don’t know if the 15 mill figure is accurate but whatever), that’s 2700 people EA let die in order to purchase a manor house. Or what about the fund that gave $28000 dollars to print out harry potter fanfiction and give it to math olympians? That’s 6 dead children sacrificed for printouts of freely available fiction!
I hope you see why I don’t like this type of rhetoric.
[Warning: long comment] Thanks for the pushback. I think converting to lives is good in other cases, especially if it’s (a) useful for judging effectiveness, and (b) not used as a misleading rhetorical device [1].
The basic point I want to make is that all interventions have to pencil out. When donating, we are trying to maximize the good we create, not decide which superficially sounds better between the different strategies “empower beneficiaries to invest in their communities’ infrastructure” and “use RCTs to choose lifesaving interventions” [2]. Lives are at stake, and I don’t think those lives are less important simply because it’s harder to put names and faces to the ~60 lives that were saved from a 0.04% chance of reduction of malaria deaths from a malaria net. Of course this applies equally to the Wytham Abbey purchase or anything else. But to point (a), we actually can compare the welfare gain from 61 lives saved to the economic security produced by this project. GiveWell has weights for doubling of consumption, partly based on interviews from Africans [3]. With other projects, this might be intractable due to entirely different cause areas or different moral preferences e.g. longtermism.
Imagine that we have a cost-effectiveness analysis made by a person with knowledge of local conditions and local moral preferences, domain expertise in East African agricultural markets, and the quantitative expertise of GiveWell analysts. If it comes out that one intervention is 5 or 10 times better than the other, as is very common, we need a very compelling reason why some consideration was missed to justify funding the other one. Compare this to our currently almost complete state of ignorance as to the value of building this plant, and you see the value of numbers. We might not get a CEA this good, but we should get close as we have all the pieces.
As to point (b), I am largely pro making these comparisons in most cases just to remind people of the value of our resources. But I feel like the Wytham and HPMOR cases, depending on phrasing, could exploit peoples’ tendency to think of projects that save lives in emotionally salient ways as better than projects that save lives via less direct methods. It will always sound bad to say that intervention A is funded rather than saving X lives, and we should generally not shut down discussion of A by creating indignation. This kind of misleading rhetoric is not at all my intention; we all understand that allowing a large enough number of farmers access to sorghum markets can produce more welfare than preventing 61 deaths from malaria. We have the choice between saving 61 of someones’ sons and daughters, and allowing X extremely poor people to perhaps buy metal roofs, send their children to school, and generally have some chance of escaping a millennia-long poverty trap. We should think: “I really want to know how large X is”.
[1] and maybe (c) not bad for your mental health?
[2] Unless you believe empowering people is inherently better regardless of the relative cost, which I strongly disagree with.
[3] This is important—Westerners may be biased here because we place different values on life compared to doubling consumption. But these interviews were from Kenya and Ghana, so maybe Uganda’s weights slightly differ.
@Thomas Kwa in my eyes this is a hugely insightful (perhaps even spectacular) response, thanks for taking the time to think about it and write it. Perhaps consder writing a full post with these kinds of insights about benefits of CEAs.
That is If you can stomach spending more time away from your real job making sure that we still exist in 50 years to even talk about GHD ;).
I think that kind of thinking is appropriate in all these cases. The Whytham abbey purchase was an investment, but it is reasonable to compare the cost compared to other investments in these terms.
On the one hand, I am sympathetic to the differential scrutiny applied to different projects.
On the other hand, there are conceptual models in which this scrutiny makes sense. In one of them, money has been pre-committed by donors ex ante to cause areas. So money in the global health/development (GH/D) bucket (or maybe the neartermism bucket) is only competing for funding against projects in the same bucket (while the HPMOR and Wytham projects were in something like a “longtermism community growth” (LCG) bucket).
As a practical matter, this is approximately true—in the context of an appeal on the Forum for funds, Anthony’s project is very likely to receive funds that would counterfactually have gone to other GH/D work (and there’s a good chance they would have counterfactually gone to GiveWell-style work). In contrast, the odds of someone like me giving money to HP fanfic distribution or Wytham are ~0.
Relatedly, one could see the GH/D and LCG fields as too methodologically different for this kind of comparison to be fitting. There’s a certain appeal to that, since the numbers in LCG cost-effectiveness analyses tend to be much less grounded in data than GH/D numbers. And if you make the “number of future lives potentially saved” high enough, the LCG project will always win even if the “chance of preventing catastrophe” is miniscule indeed. Of course, one extension to this approach would be to argue that (e.g.) EA-style GH/D projects and non-EA-style GH/D projects are also too methodologically different for this kind of analysis—and that would leave us with no common yardstick to evaluate projects in GH/D itself.
I understand the sentiment but disagree. For global health interventions, cost effectiveness analysis is doable and adds value. Most CE orgs and other aspirational cost effective orgs like my own have done some form of CEA as part of making their case
For Whytham Abby and Harry potter fan fiction that may be more difficult to do.
Although I think there should be far more CEAs across all fields. Like for Harry Potter giveout I would have done something like (obviously this is full hack)...
DISCLAIMER: I’m not saying this actual BOTEC is meaningful, I’m just giving it as an in example that these kind of CEAs are possible.
650 Students given books, I assume that of these 100-300 of these students will read them, and 0-5 will change their life trajectory towards giving or altruism to a small degree counterfactually due to the book. 0 to 2 will give 50,000 to 500,000 more over their lifetime and 0 to 5 will change their life direction and save 1 more life than they would have otherwise.
So cost effectivenss might be between 0 x $50,000 = 0 raised + 0 x 1 = 0 lives saved and 500,000 x 2 = 1,000,000 dollars and 1 x 5 lives saved.
So the book handout might be somewhere between completely useless and raising $1,000,000 extra dollars for EA causes and saving 5 lives (I don’t stand by this analysis, it’s just a brief hack)
So compared with saving 6 children with nets it might be comparable-ish based on my 2 minute math. This kind of math might also have been done and not shared in the grant review!
Again, a disclaimer that I’m not trying to justify the grant here, just mocking up the basic mechanics of the kind of botec you could do.
I’m fine with CEA’s, my problem is that this seems to have been trotted out selectively in order to dismiss Anthony’s proposal in particular, even though EA discusses and sometimes funds proposals that make the supposed “16 extra deaths” look like peanuts by comparison.
The Wytham abbey project has been sold, so we know it’s overall impact was to throw something like a million pounds down the drain (when you factor in stamp duty, etc). I think it’s deeply unfair to frame Anthony’s proposal as possibly letting 16 people die, while not doing the same for Wytham, which (in this framing) definitively let 180 people die.
Also, the cost effectiveness analysis hasn’t even been done yet! I find it kind of suspect that this is getting such a hostile response when EA insiders propose ineffective projects all the time with much less pushback. There are also differing factors here worth considering, like helping EA build links with grassroots orgs, indirectly spreading EA ideas to organisers in the third world, etc. EA spends plenty of money on “community building”, would this not count?
The HPMOR thing is a side note, but I vehemently disagree with your analysis, and the initial grant, because the counterfactual in this case is not doing nothing, it’s sending them a link to the website where HPMOR is hosted for free for everybody, which costs nothing. Plus HPMOR only tangentially advocates for EA causes anyway! A huge number of people have read HPMOR, and only a small proportion have gone on to become EA members. Your numbers are absurdly overoptimistic.
Disclaimer I don’t know much about the HPMOR thing—for example I didn’t know it only tangentially plugged EA. I was just giving a 2 minute example of the kind of analysis you might do (obviously with better info then I had), and that it is possible to do that CEA. I wasn’t trying to justify the grant at all my apologies if it came across that way!
Also I don’t think this post is getting that hostile a response?
I think everyone agrees that it’s harder to do cost effectiveness analysis for speculative projects than it is to do it for disease prevention, and that any longtermist cost/benefit analysis is going to have a lot more scope for debate on the numbers. But it is also harder to do cost effectiveness analysis in terms of lives saved for other GHD measures like rural poverty alleviation (though if this project affects malnutrition it might actually be amenable to GiveWell style analysis. )
I think ultimately if every marginal dollar proposed to be spent on GHD has to demonstrate reasoning as to why its as good as or better than AMF at the margin, it’s only fair to demand similar transparency for community building and longtermist initiatives (with an acceptance of wider error bars).[1] Especially since there’s a marked tendency for the former to be outsider organizations and the latter to be organizations within the EA network...
I make no comment either way about the particular viability of this project. And I’d actually be quite interested in your more detailed thoughts on it, as whilst you’re not an expert on farming you clearly have in depth knowledge of Uganda.
At the risk of boring on about Wytham, the bar seemed to be that it was net positive given lots of OpenPhil money was being directed to conference venues, not that it was better than buying a marginally inferior venue for a lot less money and donating the rest to initiatives that could save lives
This comment shows the challenge of the agreevote/disagreevote system:
I agree with the direction toward showing more cost-effectiveness analyses in other fields versus reducing their importance in global health/development.
I do not think the fanfic CEA is plausible, even at the 2-minute level, for some of the reasons identified by @titotal. That being said, the people funding fanfic distribution were probably interested in “people drawn to AI safety / x-risk mitigation” as their outcome variable / theory of impact, not donations made or individual lives saved. So the BOTEC is simultaneous too kind to and too demanding of that project.
Perhaps then in this case you just don’t agree or disagree when it’s 50/50? Looking at it now I also don’t think my little CEA is plausible, I do think it perhaps got taken a bit too seriously though :D!
Yeah, I didn’t vote either way, which is fine. I’m just confused about how to interpret the votes of those who did! Did they agree/disagree on both parts, or vote based on which part they thought was primary?
I don’t like that this “converting to lives” thing is being done on this kind of post and seemingly nowhere else?
Like, if we applied it to the wytham abbey purchase (I don’t know if the 15 mill figure is accurate but whatever), that’s 2700 people EA let die in order to purchase a manor house. Or what about the fund that gave $28000 dollars to print out harry potter fanfiction and give it to math olympians? That’s 6 dead children sacrificed for printouts of freely available fiction!
I hope you see why I don’t like this type of rhetoric.
[Warning: long comment] Thanks for the pushback. I think converting to lives is good in other cases, especially if it’s (a) useful for judging effectiveness, and (b) not used as a misleading rhetorical device [1].
The basic point I want to make is that all interventions have to pencil out. When donating, we are trying to maximize the good we create, not decide which superficially sounds better between the different strategies “empower beneficiaries to invest in their communities’ infrastructure” and “use RCTs to choose lifesaving interventions” [2]. Lives are at stake, and I don’t think those lives are less important simply because it’s harder to put names and faces to the ~60 lives that were saved from a 0.04% chance of reduction of malaria deaths from a malaria net. Of course this applies equally to the Wytham Abbey purchase or anything else. But to point (a), we actually can compare the welfare gain from 61 lives saved to the economic security produced by this project. GiveWell has weights for doubling of consumption, partly based on interviews from Africans [3]. With other projects, this might be intractable due to entirely different cause areas or different moral preferences e.g. longtermism.
Imagine that we have a cost-effectiveness analysis made by a person with knowledge of local conditions and local moral preferences, domain expertise in East African agricultural markets, and the quantitative expertise of GiveWell analysts. If it comes out that one intervention is 5 or 10 times better than the other, as is very common, we need a very compelling reason why some consideration was missed to justify funding the other one. Compare this to our currently almost complete state of ignorance as to the value of building this plant, and you see the value of numbers. We might not get a CEA this good, but we should get close as we have all the pieces.
As to point (b), I am largely pro making these comparisons in most cases just to remind people of the value of our resources. But I feel like the Wytham and HPMOR cases, depending on phrasing, could exploit peoples’ tendency to think of projects that save lives in emotionally salient ways as better than projects that save lives via less direct methods. It will always sound bad to say that intervention A is funded rather than saving X lives, and we should generally not shut down discussion of A by creating indignation. This kind of misleading rhetoric is not at all my intention; we all understand that allowing a large enough number of farmers access to sorghum markets can produce more welfare than preventing 61 deaths from malaria. We have the choice between saving 61 of someones’ sons and daughters, and allowing X extremely poor people to perhaps buy metal roofs, send their children to school, and generally have some chance of escaping a millennia-long poverty trap. We should think: “I really want to know how large X is”.
[1] and maybe (c) not bad for your mental health?
[2] Unless you believe empowering people is inherently better regardless of the relative cost, which I strongly disagree with.
[3] This is important—Westerners may be biased here because we place different values on life compared to doubling consumption. But these interviews were from Kenya and Ghana, so maybe Uganda’s weights slightly differ.
@Thomas Kwa in my eyes this is a hugely insightful (perhaps even spectacular) response, thanks for taking the time to think about it and write it. Perhaps consder writing a full post with these kinds of insights about benefits of CEAs.
That is If you can stomach spending more time away from your real job making sure that we still exist in 50 years to even talk about GHD ;).
I think that kind of thinking is appropriate in all these cases. The Whytham abbey purchase was an investment, but it is reasonable to compare the cost compared to other investments in these terms.
On the one hand, I am sympathetic to the differential scrutiny applied to different projects.
On the other hand, there are conceptual models in which this scrutiny makes sense. In one of them, money has been pre-committed by donors ex ante to cause areas. So money in the global health/development (GH/D) bucket (or maybe the neartermism bucket) is only competing for funding against projects in the same bucket (while the HPMOR and Wytham projects were in something like a “longtermism community growth” (LCG) bucket).
As a practical matter, this is approximately true—in the context of an appeal on the Forum for funds, Anthony’s project is very likely to receive funds that would counterfactually have gone to other GH/D work (and there’s a good chance they would have counterfactually gone to GiveWell-style work). In contrast, the odds of someone like me giving money to HP fanfic distribution or Wytham are ~0.
Relatedly, one could see the GH/D and LCG fields as too methodologically different for this kind of comparison to be fitting. There’s a certain appeal to that, since the numbers in LCG cost-effectiveness analyses tend to be much less grounded in data than GH/D numbers. And if you make the “number of future lives potentially saved” high enough, the LCG project will always win even if the “chance of preventing catastrophe” is miniscule indeed. Of course, one extension to this approach would be to argue that (e.g.) EA-style GH/D projects and non-EA-style GH/D projects are also too methodologically different for this kind of analysis—and that would leave us with no common yardstick to evaluate projects in GH/D itself.
I understand the sentiment but disagree. For global health interventions, cost effectiveness analysis is doable and adds value. Most CE orgs and other aspirational cost effective orgs like my own have done some form of CEA as part of making their case
For Whytham Abby and Harry potter fan fiction that may be more difficult to do.
Although I think there should be far more CEAs across all fields. Like for Harry Potter giveout I would have done something like (obviously this is full hack)...
DISCLAIMER: I’m not saying this actual BOTEC is meaningful, I’m just giving it as an in example that these kind of CEAs are possible.
650 Students given books, I assume that of these 100-300 of these students will read them, and 0-5 will change their life trajectory towards giving or altruism to a small degree counterfactually due to the book. 0 to 2 will give 50,000 to 500,000 more over their lifetime and 0 to 5 will change their life direction and save 1 more life than they would have otherwise.
So cost effectivenss might be between 0 x $50,000 = 0 raised + 0 x 1 = 0 lives saved and 500,000 x 2 = 1,000,000 dollars and 1 x 5 lives saved.
So the book handout might be somewhere between completely useless and raising $1,000,000 extra dollars for EA causes and saving 5 lives (I don’t stand by this analysis, it’s just a brief hack)
So compared with saving 6 children with nets it might be comparable-ish based on my 2 minute math. This kind of math might also have been done and not shared in the grant review!
Again, a disclaimer that I’m not trying to justify the grant here, just mocking up the basic mechanics of the kind of botec you could do.
I’m fine with CEA’s, my problem is that this seems to have been trotted out selectively in order to dismiss Anthony’s proposal in particular, even though EA discusses and sometimes funds proposals that make the supposed “16 extra deaths” look like peanuts by comparison.
The Wytham abbey project has been sold, so we know it’s overall impact was to throw something like a million pounds down the drain (when you factor in stamp duty, etc). I think it’s deeply unfair to frame Anthony’s proposal as possibly letting 16 people die, while not doing the same for Wytham, which (in this framing) definitively let 180 people die.
Also, the cost effectiveness analysis hasn’t even been done yet! I find it kind of suspect that this is getting such a hostile response when EA insiders propose ineffective projects all the time with much less pushback. There are also differing factors here worth considering, like helping EA build links with grassroots orgs, indirectly spreading EA ideas to organisers in the third world, etc. EA spends plenty of money on “community building”, would this not count?
The HPMOR thing is a side note, but I vehemently disagree with your analysis, and the initial grant, because the counterfactual in this case is not doing nothing, it’s sending them a link to the website where HPMOR is hosted for free for everybody, which costs nothing. Plus HPMOR only tangentially advocates for EA causes anyway! A huge number of people have read HPMOR, and only a small proportion have gone on to become EA members. Your numbers are absurdly overoptimistic.
I also probably “disagree” with my analysis
Disclaimer I don’t know much about the HPMOR thing—for example I didn’t know it only tangentially plugged EA. I was just giving a 2 minute example of the kind of analysis you might do (obviously with better info then I had), and that it is possible to do that CEA. I wasn’t trying to justify the grant at all my apologies if it came across that way!
Also I don’t think this post is getting that hostile a response?
I think everyone agrees that it’s harder to do cost effectiveness analysis for speculative projects than it is to do it for disease prevention, and that any longtermist cost/benefit analysis is going to have a lot more scope for debate on the numbers. But it is also harder to do cost effectiveness analysis in terms of lives saved for other GHD measures like rural poverty alleviation (though if this project affects malnutrition it might actually be amenable to GiveWell style analysis. )
I think ultimately if every marginal dollar proposed to be spent on GHD has to demonstrate reasoning as to why its as good as or better than AMF at the margin, it’s only fair to demand similar transparency for community building and longtermist initiatives (with an acceptance of wider error bars).[1] Especially since there’s a marked tendency for the former to be outsider organizations and the latter to be organizations within the EA network...
I make no comment either way about the particular viability of this project. And I’d actually be quite interested in your more detailed thoughts on it, as whilst you’re not an expert on farming you clearly have in depth knowledge of Uganda.
At the risk of boring on about Wytham, the bar seemed to be that it was net positive given lots of OpenPhil money was being directed to conference venues, not that it was better than buying a marginally inferior venue for a lot less money and donating the rest to initiatives that could save lives
This comment shows the challenge of the agreevote/disagreevote system:
I agree with the direction toward showing more cost-effectiveness analyses in other fields versus reducing their importance in global health/development.
I do not think the fanfic CEA is plausible, even at the 2-minute level, for some of the reasons identified by @titotal. That being said, the people funding fanfic distribution were probably interested in “people drawn to AI safety / x-risk mitigation” as their outcome variable / theory of impact, not donations made or individual lives saved. So the BOTEC is simultaneous too kind to and too demanding of that project.
Perhaps then in this case you just don’t agree or disagree when it’s 50/50? Looking at it now I also don’t think my little CEA is plausible, I do think it perhaps got taken a bit too seriously though :D!
Yeah, I didn’t vote either way, which is fine. I’m just confused about how to interpret the votes of those who did! Did they agree/disagree on both parts, or vote based on which part they thought was primary?
Thanks so much titotal.