Thanks for writing this up! This does seem to be an important argument not made often enough.
To my knowledge this has been covered a couple of times before, although not as thoroughly.
Once by Oxford Prioritization Project however they approached it from the other end, instead asking “what absolute percentage x-risk reduction would you need to get for £10,000 for it to be as cost effective as AMF” and finding the answer of 4 x 10^-8%. I think your model gives £10,000 as reducing x-risk by 10^-9%, which fits with your conclusion of close but not quite as good as global poverty.
Note they use 5% before 2100 as their risk, also do not consider QALYs, instead only looking at ‘lives saved’ which is likely bias them against AMF, since it mostly saves children.
We also calculated this as part of the Causal Networks Model I worked on with Denise Melchin at CEA over the summer. The conclusion is mentioned briefly here under ‘existential effectiveness’.
I think our model was basically the same as yours, although we were explicitly interested in the chance of existential risk before 2050, and did not include probabilistic elements. We also tried to work in QALYs, although most of our figures were more bullish than yours. We used by default:
7% chance of existential risk by 2050, which in retrospect seems extremely high, but I think was based on a survey from a conference.
The world population in 2050 will be 9.8 Billion, and each death will be worth −25 QALYs (so 245 billion QALYs at stake, very similar to yours)
For the effectiveness of research, we assumed that 10,000 researchers working for 10 years would reduce x-risk by 1% point (i.e. from 7% to 6%). We also (unreasonably) assumed each researcher year cost £50,000 (where I think the true number should be at least double that, if not much more).
Our model then had various other complicated effects, modelling both ‘theoretical’ and ‘practical’ x-risk based on government/industry willingness to use the advances, but these were second order and can mostly be ignored.
Ignoring these second order effects then, our model suggested it would cost £5 billion to reduce x-risk by 1% point, which corresponds to a cost of about £2 per QALY. In retrospect this should be at least 1 or 2 orders of magnitude higher (increasing researcher cost and decreasing x-risk possibility by and order of magnitude each).
I find your x-risk chance somewhat low, I think 5% before 2100 seems more likely. Your cost-per-percent to reduce x-risk also works out as much higher than the one we used, but seems more justified (ours was just pulled from the air as ‘reasonable sounding’).
I also think x-risk over the century is over 1%, and we can reduce it much more cheaply than your guess, though it’s nice to show it’s plausible even with conservative figures.
Thanks for writing this up! This does seem to be an important argument not made often enough.
To my knowledge this has been covered a couple of times before, although not as thoroughly.
Once by Oxford Prioritization Project however they approached it from the other end, instead asking “what absolute percentage x-risk reduction would you need to get for £10,000 for it to be as cost effective as AMF” and finding the answer of 4 x 10^-8%. I think your model gives £10,000 as reducing x-risk by 10^-9%, which fits with your conclusion of close but not quite as good as global poverty.
Note they use 5% before 2100 as their risk, also do not consider QALYs, instead only looking at ‘lives saved’ which is likely bias them against AMF, since it mostly saves children.
We also calculated this as part of the Causal Networks Model I worked on with Denise Melchin at CEA over the summer. The conclusion is mentioned briefly here under ‘existential effectiveness’.
I think our model was basically the same as yours, although we were explicitly interested in the chance of existential risk before 2050, and did not include probabilistic elements. We also tried to work in QALYs, although most of our figures were more bullish than yours. We used by default:
7% chance of existential risk by 2050, which in retrospect seems extremely high, but I think was based on a survey from a conference.
The world population in 2050 will be 9.8 Billion, and each death will be worth −25 QALYs (so 245 billion QALYs at stake, very similar to yours)
For the effectiveness of research, we assumed that 10,000 researchers working for 10 years would reduce x-risk by 1% point (i.e. from 7% to 6%). We also (unreasonably) assumed each researcher year cost £50,000 (where I think the true number should be at least double that, if not much more).
Our model then had various other complicated effects, modelling both ‘theoretical’ and ‘practical’ x-risk based on government/industry willingness to use the advances, but these were second order and can mostly be ignored.
Ignoring these second order effects then, our model suggested it would cost £5 billion to reduce x-risk by 1% point, which corresponds to a cost of about £2 per QALY. In retrospect this should be at least 1 or 2 orders of magnitude higher (increasing researcher cost and decreasing x-risk possibility by and order of magnitude each).
I find your x-risk chance somewhat low, I think 5% before 2100 seems more likely. Your cost-per-percent to reduce x-risk also works out as much higher than the one we used, but seems more justified (ours was just pulled from the air as ‘reasonable sounding’).
I also made a very rough estimate in this article: https://80000hours.org/articles/extinction-risk/#in-total-how-effective-is-it-to-reduce-these-risks Though this estimate is much better and I’ve added a link to it.
I also think x-risk over the century is over 1%, and we can reduce it much more cheaply than your guess, though it’s nice to show it’s plausible even with conservative figures.