Thanks for the post, and welcome to the EA Forum, Adam!
our results suggest that an extremely natural and seemingly obvious practice – presenting risk mitigation options individually for comparison – is suboptimal. Put bluntly: while people are relatively good at choosing individual items for their grocery carts given each item’s price per ounce (a linear allocation problem), people are less good at the nonlinear allocation problem of choosing individual existential risk interventions based on each intervention’s risk reduction per dollar spent. As a result, it may be better to instead present decision makers with competing total allocation profiles (specifying how much is to be invested in each risk), where each profile is explicitly labeled with its estimated reduction in overall existential risk (the risk that any catastrophe occurs).
The initial probability of survival in your example is (1 − 0.98)*(1 − 0.91)*(1 − 0.57) = 0.0774 %, which is pretty low[1]. In contrast, I think the probability of surviving human extinction nearterm is close to 100 %, with all potential causes being very unlikely to lead to it. So I would estimate the increase in the probability of surviving human extinction adding the decrease in the probability of each potential cause (because the other terms with multiple factors are much smaller). For example, “decrease in the probability of A” + “decrease in the probability of B” + “decrease in the probability of C”, which is a linear allocation. As a result, neglecting diminishing returns decreasing each risk in isolation (as I think you assume), I would decrease the risk of human extinction spending all the money decreasing the risk for which “reduction in the risk of human extinction in pp”/”cost” is the highest.
Thanks, Vasco. I agree with your nice point that when all risks are very close to zero, the counterintuitive phenomenon we talk about in the paper makes very little difference. (Toby Ord also notes this in The Precipice (Appendix D).)
Thanks, Adam. The risks do not even have to be very close to 0. 10 % is enough. For 2 risks r1 and r2, the probability of survival is p_s_i = (1 - r1)*(1 - r2). If the risks change by Delta_r1 and Delta_r2, the probability of survival becomes p_s_f = (1 - r1 - Delta_r1)*(1 - r2 - Delta_r2) = p_s - (1 - r2)*Delta_r1 - (1 - r1)*Delta_r2 + Delta_r1*Delta_r2. So the increase in the probability of survival is Delta_p_s = p_s_f—p_s_i = -(1 - r2)*Delta_r1 - (1 - r1)*Delta_r2 + Delta_r1*Delta_r2. If both risks are at most 10 %, which I would say very much holds in reality, 1 - r1 and 1 - r2 are roughly 1. In addition, for relatively small allocations of money, the last term is negligible. So the increase in the probability of survival is approximately -Delta_r1 - Delta_r2, which is the sum of the decreases in risks 1 and 2.
I like your idea of doing a back-of-the-envelope calculation assuming that there are just two risks. Suppose that risk #1 has survival probability s1 and risk #2 has survival probability s2. Assume s1 < s2. Let’s compare two alternative interventions:
Intervention 1: Increase the survival probability for risk #1 by some small amount x.
Intervention 2: Increase the survival probability for risk #2 by that same amount x.
Now we have:
Intervention 1 will increase the overall survival probability from s1*s2 to (s1+x)*s2. That is an increase of x*s2.
Intervention 2 will increase the overall survival probability from s1*s2 to s1*(s2+x). That is an increase of x*s1.
So the increase in overall survival probability produced by intervention #1 is 100 * (s2/s1 − 1) percent greater than the increase in overall survival probability produced by intervention #2.
Plugging in some toy numbers:
Suppose that risk #1 has survival probability 90% and risk #2 has survival probability 99%. Then an intervention that increases s1 by a small amount will produce 10% more increase in overall survival probability than an intervention that increases s2 by that same small amount.
If risk #1 had survival probability 80% instead, then intervention 1 would produce approximately 24% more increase in overall survival probability than intervention 2.
So a risk’s base survival probability does make some difference to how much one increases overall survival probability by mitigating that risk. But overall I agree with you that for risks with survival probability greater than 90%, the difference is modest.
Thanks for the post, and welcome to the EA Forum, Adam!
The initial probability of survival in your example is (1 − 0.98)*(1 − 0.91)*(1 − 0.57) = 0.0774 %, which is pretty low[1]. In contrast, I think the probability of surviving human extinction nearterm is close to 100 %, with all potential causes being very unlikely to lead to it. So I would estimate the increase in the probability of surviving human extinction adding the decrease in the probability of each potential cause (because the other terms with multiple factors are much smaller). For example, “decrease in the probability of A” + “decrease in the probability of B” + “decrease in the probability of C”, which is a linear allocation. As a result, neglecting diminishing returns decreasing each risk in isolation (as I think you assume), I would decrease the risk of human extinction spending all the money decreasing the risk for which “reduction in the risk of human extinction in pp”/”cost” is the highest.
So low that it is unclear one should try to avoid the catastrophes?
Thanks, Vasco. I agree with your nice point that when all risks are very close to zero, the counterintuitive phenomenon we talk about in the paper makes very little difference. (Toby Ord also notes this in The Precipice (Appendix D).)
Thanks, Adam. The risks do not even have to be very close to 0. 10 % is enough. For 2 risks r1 and r2, the probability of survival is p_s_i = (1 - r1)*(1 - r2). If the risks change by Delta_r1 and Delta_r2, the probability of survival becomes p_s_f = (1 - r1 - Delta_r1)*(1 - r2 - Delta_r2) = p_s - (1 - r2)*Delta_r1 - (1 - r1)*Delta_r2 + Delta_r1*Delta_r2. So the increase in the probability of survival is Delta_p_s = p_s_f—p_s_i = -(1 - r2)*Delta_r1 - (1 - r1)*Delta_r2 + Delta_r1*Delta_r2. If both risks are at most 10 %, which I would say very much holds in reality, 1 - r1 and 1 - r2 are roughly 1. In addition, for relatively small allocations of money, the last term is negligible. So the increase in the probability of survival is approximately -Delta_r1 - Delta_r2, which is the sum of the decreases in risks 1 and 2.
I like your idea of doing a back-of-the-envelope calculation assuming that there are just two risks. Suppose that risk #1 has survival probability s1 and risk #2 has survival probability s2. Assume s1 < s2. Let’s compare two alternative interventions:
Intervention 1: Increase the survival probability for risk #1 by some small amount x.
Intervention 2: Increase the survival probability for risk #2 by that same amount x.
Now we have:
Intervention 1 will increase the overall survival probability from s1*s2 to (s1+x)*s2. That is an increase of x*s2.
Intervention 2 will increase the overall survival probability from s1*s2 to s1*(s2+x). That is an increase of x*s1.
So the increase in overall survival probability produced by intervention #1 is 100 * (s2/s1 − 1) percent greater than the increase in overall survival probability produced by intervention #2.
Plugging in some toy numbers:
Suppose that risk #1 has survival probability 90% and risk #2 has survival probability 99%. Then an intervention that increases s1 by a small amount will produce 10% more increase in overall survival probability than an intervention that increases s2 by that same small amount.
If risk #1 had survival probability 80% instead, then intervention 1 would produce approximately 24% more increase in overall survival probability than intervention 2.
So a risk’s base survival probability does make some difference to how much one increases overall survival probability by mitigating that risk. But overall I agree with you that for risks with survival probability greater than 90%, the difference is modest.
Thanks for the elaboration, Adam! That makes sense.