Thanks again for writing this. I just wanted to flag a potential issue with the $125 to $1,250 per human-life-equivalent-saved figure for ‘x-risk prevention.’
I think that figure is based on a willingness-to-pay proposal that already assumes some kind of longtermism.
You base the range Linch’s proposal of aiming to reducing x-risk by 0.01% per $100m-$1bn. As far as I can tell, these figures are based on a rough proposal of what we should be willing to pay for existential risk reduction: Linch refers to this post on “How many EA 2021 $s would you trade off against a 0.01% chance of existential catastrophe?”, which includes the proposed answer “we should fund interventions that we have resilient estimates of reducing x-risk ~0.01% at a cost of ~$100M.”
But I think that the willingness to pay from Linch is based on accounting for future lives, rather than the kind of currently-alive-human-life-equivalent-saved figure that you’re looking for. (@Linch, please do correct me if I’m wrong!)
In short, saying that we should fund interventions at the $100m/0.01% bar doesn’t say whether there are many (or any) available interventions at that level of cost-effectiveness. And while I appreciate that some grantmakers have begun leaning more on that kind of quantitative heuristic, I don’t doubt that you can infer from this fact that previously or currently funded work on ‘general x-risk prevention’ has met that bar, or even come particularly close to it.
So, I think the $125-$1,250 figure already assumes longtermism and isn’t applicable to your question. (Though I may have missed something here and would be happy to stand correct – particularly if I have misrepresented Linch’s analysis!)
Of course, if the upshot is that ‘general x-risk prevention’ is less cost-effective than the $125-$1,250 per currently-alive-human-life-equivalent-saved, then your overall point only becomes even stronger.
(PS: As an aside, I think it would be a good practice to add some kind of caption beneath your table stating how these are rough estimates, and perhaps in some cases even the only available estimate for that quantity. I’m pretty concerned about long citation trails in longtermist analysis, where very influential claims sometimes bottom out to some extremely rough and fragile estimates. Given how rough these estimates are, I think it’d be better if others replicated their own analysis from scratch before citing them.)
I think the numbers work out assuming x-risk means (almost) everyone being killed and the percent reduction is absolute (percentage point), not relative:
My concern is not that the numbers don’t work out. My concern is that the “$100m/0.01%” figure is not an estimate of how cost-effective ‘general x-risk prevention’ actually is in the way that this post implies.
It’s not an empirical estimate, it’s a proposed funding threshold, i.e. an answer to Linch’s question “How many EA 2021 $s would you trade off against a 0.01% chance of existential catastrophe?” But saying that we should fund interventions at that level of cost-effectiveness doesn’t say whether are many (or any) such interventions available at the moment. If I say “I propose that GiveWell should endorse interventions that we expect to save a life per $500”, that doesn’t by itself show whether such interventions exist.
Of course, the proposed funding threshold could be informed by cost-effectiveness estimates for specific interventions; I actually suspect that it is. But then it would be useful to see those estimates – or at the very least know which interventions they are – before establishing that figure as the ‘funding bar’ in this analysis.
This is particularly relevant if those estimates are based on interventions that do not prevent catastrophic events but merely prevent them from reaching existential/extinction levels, as the latter category does not affect all currently living people, meaning that ‘8 billion people’ would be the wrong number for the estimation you wrote above.
I agree with your points. I was responding to this point, but should have quoted it to be clearer:
“But I think that the willingness to pay from Linch is based on accounting for future lives, rather than the kind of currently-alive-human-life-equivalent-saved figure that you’re looking for.”
I think the numbers can work out without considering future lives or at least anything other than deaths.
But I think that the willingness to pay from Linch is based on accounting for future lives, rather than the kind of currently-alive-human-life-equivalent-saved figure that you’re looking for. (@Linch, please do correct me if I’m wrong!)
I think the understanding is based on how many $$s the longtermist/x-risk portion of EAs have access to, and then trying to rationally allocate resources according to that constraint. I’m not entirely sure what you mean by “accounting for future lives,” but yes, there’s an implicit assumption that under no realistic ranges of empirical uncertainty would it make sense to e.g. donate to AMF over longtermist interventions.
A moderate penalty to my numbers (from a presentist lens) is that at least some of the interventions I’m most excited about on the margin are from a civilizational resilience/recovery angle. However, I don’t think this is a large effectiveness penalty, since many other people are similarly or much more excited on the margin about AI risk interventions (which has much more the property that either approximately everybody dies or approximately no one dies).
So, I don’t think elifland’s analysis here is clearly methodologically wrong. Even though my numbers (and other analysis like mine) were based on the assumption that longtermist $$s were used for longtermist goals, it could still be the case that they are more effective for preventing deaths of existing people than existing global health interventions are. At least first order, it should not be that surprising. That is, global health interventions were chosen from the constraint of the first existing interventions with a large evidential base, whereas global catastrophic risk and existential-risk reducing interventions were chosen from (among others) the basis of dialing back ambiquity aversion and weirdness aversion to close to zero.
I think the main question/crux is how much you want to “penalize for (lack of) rigor.” Givewell-style analysis have years of dedicated work put into them. Much of my gut pulls grew out of an afternoon of relatively clear thinking (and then maybe a few more days of significantly-lower-quality thinking and conversations, etc, that adjusted my numbers somewhat but not hugely). I never really understood the principled solutions to problems like the optimizer’s curse and suspicious convergence.
PS: As an aside, I think it would be a good practice to add some kind of caption beneath your table stating how these are rough estimates, and perhaps in some cases even the only available estimate for that quantity. I’m pretty concerned about long citation trails in longtermist analysis, where very influential claims sometimes bottom out to some extremely rough and fragile estimates.
PS: As an aside, I think it would be a good practice to add some kind of caption beneath your table stating how these are rough estimates, and perhaps in some cases even the only available estimate for that quantity. I’m pretty concerned about long citation trails in longtermist analysis, where very influential claims sometimes bottom out to some extremely rough and fragile estimates.
I agree! I bolded the rough in the header because I didn’t want people to take the numbers too seriously but agree that probably wasn’t enough.
I tried to add a caption to the table before posting but couldn’t figure out how; is it possible / if so how do I do it?
Thanks again for writing this. I just wanted to flag a potential issue with the $125 to $1,250 per human-life-equivalent-saved figure for ‘x-risk prevention.’
I think that figure is based on a willingness-to-pay proposal that already assumes some kind of longtermism.
You base the range Linch’s proposal of aiming to reducing x-risk by 0.01% per $100m-$1bn. As far as I can tell, these figures are based on a rough proposal of what we should be willing to pay for existential risk reduction: Linch refers to this post on “How many EA 2021 $s would you trade off against a 0.01% chance of existential catastrophe?”, which includes the proposed answer “we should fund interventions that we have resilient estimates of reducing x-risk ~0.01% at a cost of ~$100M.”
But I think that the willingness to pay from Linch is based on accounting for future lives, rather than the kind of currently-alive-human-life-equivalent-saved figure that you’re looking for. (@Linch, please do correct me if I’m wrong!)
In short, saying that we should fund interventions at the $100m/0.01% bar doesn’t say whether there are many (or any) available interventions at that level of cost-effectiveness. And while I appreciate that some grantmakers have begun leaning more on that kind of quantitative heuristic, I don’t doubt that you can infer from this fact that previously or currently funded work on ‘general x-risk prevention’ has met that bar, or even come particularly close to it.
So, I think the $125-$1,250 figure already assumes longtermism and isn’t applicable to your question. (Though I may have missed something here and would be happy to stand correct – particularly if I have misrepresented Linch’s analysis!)
Of course, if the upshot is that ‘general x-risk prevention’ is less cost-effective than the $125-$1,250 per currently-alive-human-life-equivalent-saved, then your overall point only becomes even stronger.
(PS: As an aside, I think it would be a good practice to add some kind of caption beneath your table stating how these are rough estimates, and perhaps in some cases even the only available estimate for that quantity. I’m pretty concerned about long citation trails in longtermist analysis, where very influential claims sometimes bottom out to some extremely rough and fragile estimates. Given how rough these estimates are, I think it’d be better if others replicated their own analysis from scratch before citing them.)
I think the numbers work out assuming x-risk means (almost) everyone being killed and the percent reduction is absolute (percentage point), not relative:
$100,000,000 / (0.01% * 8 billion people) = $125/person
Thanks for your reply.
My concern is not that the numbers don’t work out. My concern is that the “$100m/0.01%” figure is not an estimate of how cost-effective ‘general x-risk prevention’ actually is in the way that this post implies.
It’s not an empirical estimate, it’s a proposed funding threshold, i.e. an answer to Linch’s question “How many EA 2021 $s would you trade off against a 0.01% chance of existential catastrophe?” But saying that we should fund interventions at that level of cost-effectiveness doesn’t say whether are many (or any) such interventions available at the moment. If I say “I propose that GiveWell should endorse interventions that we expect to save a life per $500”, that doesn’t by itself show whether such interventions exist.
Of course, the proposed funding threshold could be informed by cost-effectiveness estimates for specific interventions; I actually suspect that it is. But then it would be useful to see those estimates – or at the very least know which interventions they are – before establishing that figure as the ‘funding bar’ in this analysis.
This is particularly relevant if those estimates are based on interventions that do not prevent catastrophic events but merely prevent them from reaching existential/extinction levels, as the latter category does not affect all currently living people, meaning that ‘8 billion people’ would be the wrong number for the estimation you wrote above.
I agree with your points. I was responding to this point, but should have quoted it to be clearer:
“But I think that the willingness to pay from Linch is based on accounting for future lives, rather than the kind of currently-alive-human-life-equivalent-saved figure that you’re looking for.”
I think the numbers can work out without considering future lives or at least anything other than deaths.
I think the understanding is based on how many $$s the longtermist/x-risk portion of EAs have access to, and then trying to rationally allocate resources according to that constraint. I’m not entirely sure what you mean by “accounting for future lives,” but yes, there’s an implicit assumption that under no realistic ranges of empirical uncertainty would it make sense to e.g. donate to AMF over longtermist interventions.
A moderate penalty to my numbers (from a presentist lens) is that at least some of the interventions I’m most excited about on the margin are from a civilizational resilience/recovery angle. However, I don’t think this is a large effectiveness penalty, since many other people are similarly or much more excited on the margin about AI risk interventions (which has much more the property that either approximately everybody dies or approximately no one dies).
So, I don’t think elifland’s analysis here is clearly methodologically wrong. Even though my numbers (and other analysis like mine) were based on the assumption that longtermist $$s were used for longtermist goals, it could still be the case that they are more effective for preventing deaths of existing people than existing global health interventions are. At least first order, it should not be that surprising. That is, global health interventions were chosen from the constraint of the first existing interventions with a large evidential base, whereas global catastrophic risk and existential-risk reducing interventions were chosen from (among others) the basis of dialing back ambiquity aversion and weirdness aversion to close to zero.
I think the main question/crux is how much you want to “penalize for (lack of) rigor.” Givewell-style analysis have years of dedicated work put into them. Much of my gut pulls grew out of an afternoon of relatively clear thinking (and then maybe a few more days of significantly-lower-quality thinking and conversations, etc, that adjusted my numbers somewhat but not hugely). I never really understood the principled solutions to problems like the optimizer’s curse and suspicious convergence.
Strongly agreed.
I agree! I bolded the rough in the header because I didn’t want people to take the numbers too seriously but agree that probably wasn’t enough.
I tried to add a caption to the table before posting but couldn’t figure out how; is it possible / if so how do I do it?