Thanks for doing the work to make a specific ALLFED cost effectiveness estimate! I think the AIs made a number of good points. However, I was saying the CEARCH result is ~$170 per life because GiveWell uses $5000/ālife and CEARCH was saying 30x as cost effective as GiveWell. I think GiveWell uses averting a child death means saving ~37 DALYs, and an adult death ~30 DALYs. I donāt think the AIās assumption of 80 QALYs per life saved is realistic (unless you are expecting radical life extension). The AI starts with CEARCH and then adjusts cost per life saved upward. I think there are good reasons why CEARCH is an overestimate of cost per life saved. For one, it finds nuclear risk to be significantly smaller than volcanic risk. Most analysts in this space think that the nuclear risk is significantly larger than the volcanic risk. Furthermore, the AI assumes that ALLFEDās work outside of policy for abrupt sunlight reduction scenario (ASRS) is less cost-effective than the ASRS policy work. However, I think the pandemic work is likely to be even more cost-effective, especially from the long term perspective, because pandemics are generally regarded as a greater existential risk.
The AI did seem to agree with the argument that ALLFED should have a long-term impact, it just didnāt think that the AI x-risk estimate should be used. Thatās fineāI didnāt think you would want to do a bespoke model for ALLFED, but now that you have done it for the near term, I do think it is important to do it for the long term. The AI points out that the marginal cost effectiveness calculations of the longterm impact in the journal articles are out of date because we have now spent more money. Of course thatās true, but thatās why we also calculated the cost effectiveness of spending hundreds of millions of dollars to see if the whole effort was justified. And indeed that still came out as more cost effective than AI safety. Now of course other things have changed since ~2021. AI timelines have gotten much shorter, but we were assuming that only $3 billion would be spent on AI safety, and I think itās pretty clear that a lot more than that will be spent now (especially if you count the total compensation including stock options of AI safety workers in the labs (even with your weighting of 0.3 for lab work), but that might be a topic for another post). AI 2040 hopes that trillions of dollars will be spent on AIS. Also since then, nuclear risk has gotten larger per year with the Ukraine war and potential acceleration and destabilization due to AI. Also, engineered pandemic risk per year has gone up with AI capabilities. However, this does mean a shorter number of years in expectation that the nuclear and pandemic risk might be relevant if you think the nuclear and pandemic risk will go away after AGI/āASI. For comparison, your cost per microprobability of reduction in x-risk of AI safety is $1.2 million. The median in the papers for the 3 billionth dollar on AIS was $2.5 million, with the mean being lower, so pretty good agreement with your value. So overall, since 2021, the relative marginal cost effectiveness of spending hundreds of millions of dollars on GCR resilience vs what we think will be spent on AIS I donāt think has changed too much.
The AI missed other outside evaluations of ALLFEDās longterm impact:
Speedrun: Demonstrate the ability to rapidly scale food production in the case of nuclear winter by Marie Buhl from Rethink Priorities: āmy (extremely rough) estimate that this project reduces x-risk with a cost-effectiveness of ~$260 million per 0.01% absolute reduction[1] (~70% confidence interval: 2.2 million to 2.7 billion). If this estimate were accurate, then this project would clear our median roughly estimated cost-effectiveness bar of $500M per basis-point of x-risk avertedā.
Shallow evaluations of longtermist organizations by NuƱo Sempere: āI disagree strongly with ALLFEDās estimates (probability of cost overruns, impact of ALLFEDās work if deployed, etc.), however, I feel that the case for an organization working in this area is relatively solid.ā (Note that this is a 5 year old analysis, but he recently said he respects ALLFED more now).
Iām curious if you have thoughts on how itād be if you submitted feedback like youāve done here into some sort of form, the LLMs went back and forth processing it like that, updated the site, and published a transcript like the above. I think if I make it fully automated right now itād be fairly exploitable due to LLM sycophancy, unless I tried pretty hard to mitigate that.
Interesting idea! I guess it would be less exploitable than direct edits like Wikipedia.
-
As for putting āunknownā EAs/ārationalists on the list, I see the drawback of putting a lot of them on if you are targeting a general audience. But I do think it is compelling to show that even without a long-term perspective, donating to existential risk reduction can allow everyday people to beat billionaires in terms of lives saved.
Thanks for doing an estimate of the long-term impact of ALLFED!
A marginal donation to a resilient-food organization produces about one-third as much risk reduction per dollar as Buhlās modeled pilot program. Buhl spent about 10 hours on the initial analysis and another 5-10 hours revising it after feedback from ALLFED. The model covered a hypothetical $10-$100 million project rather than a current marginal donation. Additional hazards, research, and advocacy provide some omitted upside. This assumption raises the estimated cost per microprobability by 3x as one all-things-considered adjustment.
Since ALLFED has not gotten that much money, I would argue that the marginal cost effectiveness should be higher than Buhlās estimate, not 3x lower. Indeed, we are focusing now on pilots that cost a lot less money, but still have similar impact. One example is growing plants in simulated nuclear winter conditions.
Strong work across this broader category produces half as much risk reduction per dollar as the resilient-food organization-level anchor. Resilient food is the only intervention in this category with a quantitative independent estimate of this pathway. Cross-hazard planning, infrastructure continuity, and recovery work have less direct evidence, although some interventions could be better. This assumption raises the estimated cost per microprobability by 2x.
For the impact on this century, the AI argued that policy was evaluated, but other things like resilient food pilots have not been evaluated, so it assigned an overall lower cost effectiveness. Now for the long-term future impact, the AI is arguing that resilient food pilots have been quantified, and other things like policy have not, so it is assigning a lower cost effectiveness. I think itās more defensible to assign about the same cost effectiveness across the board (we do try hard at ALLFED to equate the marginal cost effectiveness of different projects we can work on), though I agree it would be best to do a separate cost effectiveness analysis on the biosecurity work.
But the most important thing is that there is a long-term future impact of ALLFED quantified at all, so I appreciate your effort.
Thanks for doing the work to make a specific ALLFED cost effectiveness estimate! I think the AIs made a number of good points. However, I was saying the CEARCH result is ~$170 per life because GiveWell uses $5000/ālife and CEARCH was saying 30x as cost effective as GiveWell. I think GiveWell uses averting a child death means saving ~37 DALYs, and an adult death ~30 DALYs. I donāt think the AIās assumption of 80 QALYs per life saved is realistic (unless you are expecting radical life extension). The AI starts with CEARCH and then adjusts cost per life saved upward. I think there are good reasons why CEARCH is an overestimate of cost per life saved. For one, it finds nuclear risk to be significantly smaller than volcanic risk. Most analysts in this space think that the nuclear risk is significantly larger than the volcanic risk. Furthermore, the AI assumes that ALLFEDās work outside of policy for abrupt sunlight reduction scenario (ASRS) is less cost-effective than the ASRS policy work. However, I think the pandemic work is likely to be even more cost-effective, especially from the long term perspective, because pandemics are generally regarded as a greater existential risk.
The AI did seem to agree with the argument that ALLFED should have a long-term impact, it just didnāt think that the AI x-risk estimate should be used. Thatās fineāI didnāt think you would want to do a bespoke model for ALLFED, but now that you have done it for the near term, I do think it is important to do it for the long term. The AI points out that the marginal cost effectiveness calculations of the longterm impact in the journal articles are out of date because we have now spent more money. Of course thatās true, but thatās why we also calculated the cost effectiveness of spending hundreds of millions of dollars to see if the whole effort was justified. And indeed that still came out as more cost effective than AI safety. Now of course other things have changed since ~2021. AI timelines have gotten much shorter, but we were assuming that only $3 billion would be spent on AI safety, and I think itās pretty clear that a lot more than that will be spent now (especially if you count the total compensation including stock options of AI safety workers in the labs (even with your weighting of 0.3 for lab work), but that might be a topic for another post). AI 2040 hopes that trillions of dollars will be spent on AIS. Also since then, nuclear risk has gotten larger per year with the Ukraine war and potential acceleration and destabilization due to AI. Also, engineered pandemic risk per year has gone up with AI capabilities. However, this does mean a shorter number of years in expectation that the nuclear and pandemic risk might be relevant if you think the nuclear and pandemic risk will go away after AGI/āASI. For comparison, your cost per microprobability of reduction in x-risk of AI safety is $1.2 million. The median in the papers for the 3 billionth dollar on AIS was $2.5 million, with the mean being lower, so pretty good agreement with your value. So overall, since 2021, the relative marginal cost effectiveness of spending hundreds of millions of dollars on GCR resilience vs what we think will be spent on AIS I donāt think has changed too much.
The AI missed other outside evaluations of ALLFEDās longterm impact:
Speedrun: Demonstrate the ability to rapidly scale food production in the case of nuclear winter by Marie Buhl from Rethink Priorities: āmy (extremely rough) estimate that this project reduces x-risk with a cost-effectiveness of ~$260 million per 0.01% absolute reduction[1] (~70% confidence interval: 2.2 million to 2.7 billion). If this estimate were accurate, then this project would clear our median roughly estimated cost-effectiveness bar of $500M per basis-point of x-risk avertedā.
Shallow evaluations of longtermist organizations by NuƱo Sempere: āI disagree strongly with ALLFEDās estimates (probability of cost overruns, impact of ALLFEDās work if deployed, etc.), however, I feel that the case for an organization working in this area is relatively solid.ā (Note that this is a 5 year old analysis, but he recently said he respects ALLFED more now).
Interesting idea! I guess it would be less exploitable than direct edits like Wikipedia.
-
As for putting āunknownā EAs/ārationalists on the list, I see the drawback of putting a lot of them on if you are targeting a general audience. But I do think it is compelling to show that even without a long-term perspective, donating to existential risk reduction can allow everyday people to beat billionaires in terms of lives saved.
Thanks for the additional thoughts. Iāll feed in your suggestions when I get some time and make any updates that result from that.
RE: unknown EAs/ārationalists, yeah I agree having some of them on the list is valuable. I plan to add some soon if I donāt get enough volunteers.
Thanks for doing an estimate of the long-term impact of ALLFED!
Since ALLFED has not gotten that much money, I would argue that the marginal cost effectiveness should be higher than Buhlās estimate, not 3x lower. Indeed, we are focusing now on pilots that cost a lot less money, but still have similar impact. One example is growing plants in simulated nuclear winter conditions.
For the impact on this century, the AI argued that policy was evaluated, but other things like resilient food pilots have not been evaluated, so it assigned an overall lower cost effectiveness. Now for the long-term future impact, the AI is arguing that resilient food pilots have been quantified, and other things like policy have not, so it is assigning a lower cost effectiveness. I think itās more defensible to assign about the same cost effectiveness across the board (we do try hard at ALLFED to equate the marginal cost effectiveness of different projects we can work on), though I agree it would be best to do a separate cost effectiveness analysis on the biosecurity work.
But the most important thing is that there is a long-term future impact of ALLFED quantified at all, so I appreciate your effort.