For reference, here are the predictions for human extinction by 2100:
The 95 % confidence intervals for the total extinction risk by 2100 seem quite narrow. For superforecasters, domain and non-domain experts, and general x-risk experts, the upper bound is only 2, 3 and 5 times the lower bound.
It is interesting to note that, according to the superforecasters, the extinction risk from nuclear war is predicted to be 7.4 (= 0.074/0.01) times that of engineered pathogens, and 19.5 % (= 0.074/0.38) that of AI. In contrast, Toby Ord guessed in The Precipice that the existential risk from 2021 to 2120 from nuclear war is 3.33 % (= 0.1/3) that of engineered pathogens, and 1 % (= 0.1/10) that of AI.
I’m quite surprised that superforecasters predict nuclear extinction is 7.4 times more likely than engineered pandemic extinction, given that (as you suggest) EA predictions usually go the other way. Do you know if this is discussed in the paper? I had a look around and couldn’t find any discussion.
I was also curious to understand why superforecasters’ nuclear extinction risk was so high. Sources of agreement, disagreement and uncertainty, and arguments for low and high estimates are discussed on pp. 298 to 303. I checked these a few months ago, and my recollection is that the forecasters have the right qualitative considerations in mind, but I do believe they are arriving to an overly high extinction risk. I recently commented about this.
Note domain experts guessed an even higher nuclear extinction probability by 2100 of 0.55 %, 7.43 (= 0.0055/0.00074) times that of the superforecasters. This is specially surprising considering:
The pool of experts drew more heavily from the EA community than the pool of superforecasters. “The sample drew heavily from the Effective Altruism (EA) community: about 42% of experts and 9% of superforecasters reported that they had attended an EA meetup”.
I would have expected people in the EA community to guess a lower nuclear extinction risk. 0.55 % is 5.5 times Toby Ord’s guess given in The Precipice for nuclear existential risk from 2021 to 2120 of 0.1 %, and extinction risk should be lower than existential risk.
Agreed. However, the higher the uncertainty of each forecaster, the greater the variation across forecasters’ best guesses will tend to be, and therefore the wider the 95 % confidence interval of the median?
That’s true for means, where we can simply apply the CLT. However, this is a median. Stack Exchange suggests that only the density at the median matters. That means a very peaky distribution, even with wide tails will still lead to a small confidence interval. Due to forecasters rounding answers, the distribution is plausibly pretty peaky.
The confidence interval width still goes with sample size as 1/√n. There’s a decent sample size here of superforecasters.
Intuitively: you don’t care how spread out the tails are for the median, only how much of the mass is in the tails.
I was thinking as follows. The width of the confidence interval of quantile q for a confidence level alpha isF(q2 = q + z*(q*(1 - q)/n)^0.5) - F(q1 = q—z*(q*(1 - q)/n)^0.5), where P(z ⇐ X | X ~ N(0, 1)) = 1 - (1 - alpha)/2. A greater variation across estimates does not change q1 nor q2, but it increases the width F(q2) - F(q1).
That being said, I have to concede that what we care about is the mass in the tails, not the tails of the median. So one should care about the difference between e.g. the 97.5th and 2.5th percentile, not F(q2) - F(q1).
Thanks for sharing!
For reference, here are the predictions for human extinction by 2100:
The 95 % confidence intervals for the total extinction risk by 2100 seem quite narrow. For superforecasters, domain and non-domain experts, and general x-risk experts, the upper bound is only 2, 3 and 5 times the lower bound.
It is interesting to note that, according to the superforecasters, the extinction risk from nuclear war is predicted to be 7.4 (= 0.074/0.01) times that of engineered pathogens, and 19.5 % (= 0.074/0.38) that of AI. In contrast, Toby Ord guessed in The Precipice that the existential risk from 2021 to 2120 from nuclear war is 3.33 % (= 0.1/3) that of engineered pathogens, and 1 % (= 0.1/10) that of AI.
I’m quite surprised that superforecasters predict nuclear extinction is 7.4 times more likely than engineered pandemic extinction, given that (as you suggest) EA predictions usually go the other way. Do you know if this is discussed in the paper? I had a look around and couldn’t find any discussion.
Hi EJT,
I was also curious to understand why superforecasters’ nuclear extinction risk was so high. Sources of agreement, disagreement and uncertainty, and arguments for low and high estimates are discussed on pp. 298 to 303. I checked these a few months ago, and my recollection is that the forecasters have the right qualitative considerations in mind, but I do believe they are arriving to an overly high extinction risk. I recently commented about this.
Note domain experts guessed an even higher nuclear extinction probability by 2100 of 0.55 %, 7.43 (= 0.0055/0.00074) times that of the superforecasters. This is specially surprising considering:
The pool of experts drew more heavily from the EA community than the pool of superforecasters. “The sample drew heavily from the Effective Altruism (EA) community: about 42% of experts and 9% of superforecasters reported that they had attended an EA meetup”.
I would have expected people in the EA community to guess a lower nuclear extinction risk. 0.55 % is 5.5 times Toby Ord’s guess given in The Precipice for nuclear existential risk from 2021 to 2120 of 0.1 %, and extinction risk should be lower than existential risk.
.074/.01 is 7.4, not 74
Thanks! Corrected.
The confidence intervals are for what the median person in the class would forecast. Each forecaster’s uncertainty is not reflected.
Hi Joshua,
Agreed. However, the higher the uncertainty of each forecaster, the greater the variation across forecasters’ best guesses will tend to be, and therefore the wider the 95 % confidence interval of the median?
That’s true for means, where we can simply apply the CLT. However, this is a median. Stack Exchange suggests that only the density at the median matters. That means a very peaky distribution, even with wide tails will still lead to a small confidence interval. Due to forecasters rounding answers, the distribution is plausibly pretty peaky.
The confidence interval width still goes with sample size as 1/√n. There’s a decent sample size here of superforecasters.
Intuitively: you don’t care how spread out the tails are for the median, only how much of the mass is in the tails.
Thanks for following up!
I was thinking as follows. The width of the confidence interval of quantile q for a confidence level alpha is F(q2 = q + z*(q*(1 - q)/n)^0.5) - F(q1 = q—z*(q*(1 - q)/n)^0.5), where P(z ⇐ X | X ~ N(0, 1)) = 1 - (1 - alpha)/2. A greater variation across estimates does not change q1 nor q2, but it increases the width F(q2) - F(q1).
That being said, I have to concede that what we care about is the mass in the tails, not the tails of the median. So one should care about the difference between e.g. the 97.5th and 2.5th percentile, not F(q2) - F(q1).