I understand the desire to use cumulative probability to calculate probability of nuclear war before 2050, but if interdependency of base rate was not used (i.e. 0.0127 * 26 = 0.33, which is equivalent to metaculus), shouldn’t we already use a Beta conjugation of the base rate as each year pass-by?
- If detonation does not happen, Beta(1, 79)
- If detonation happen, Beta(2, 79)
- annual probability = 0.0127
- Cumulative probability of 21.843% by 2050
I saw you use Beta distribution for the CDF constraint the probability of a large nuclear war, defined using the metaculus question, I agree with this, I think this checks-out. I also like that you give less weighting to the metaculus question that ask for probability distribution as it will be less accurate than taking the Beta distribution of 100 to 1000, I learnt something about how to evaluate metaculus question here:
There seems to be 2 set of questions regarding nuclear impact and winter:
- The Nuclear Risk Horizon Project (no monetary incentive)
- Nuclear Risk Tournament ($ 2685.5 reward, and ends on 1st Feb 2024)
I wan to understand how do you calibrate the monetary incentive and limited time frame when weighing the 2 sets of questions for your research?
For example contrasting these 2 questions, which you have addressed in your post:
How many nuclear weapons will be detonated offensively by 2050, if at least one offensive detonation occurs? [HORIZON, non-monetary]
How many non-strategic nuclear weapons will be deployed at the end of 2023? (No recency weighted)? [TOURNAMENT, monetary]
The deployment mean is an orders of magnitude higher than predicted detonation. Surely, even 100 weapons is a very contained regional war scenario according to Hochmann et al (2021). And a very constrained exchange between Russia/China and NATO. I would think that the former question and prediction unrealistically low given how many test just NK have conducted recently. I think you have adequately modelled that with your beta-distribution, but that will be 3x higher than the latter question unweighted results which is about 112 at median, and 161 weapons at the 75th percentile (11 Tg soot), and the 95th percentile of your calculation of 1.81k is 3x the latter question’s distribution, do you think there’s a need to reconcile that?.
How do you feel about taking expected value of such numbers https://www.metaculus.com/questions/8382/1000-nuke-detonations-cause-4b-deaths/ (4 Bil * 0.45) when this seems so far lower than numbers proposed by more sophisticated modelling, esp the Rutgers Team. I am generally going on the heuristic on prediction market probably have an upper hand in counting weapons and predicting deployment and number and location of detonation, but not on long drag-out nuclear winter affects (crop yield, trade, famine numbers).
I still need time to engage with the soot calculation literature, so I will probably write a follow-up on that later next week or the week after if that’s okay, that will give me much more focus on asking the right questions and doing the right research.
I understand the desire to use cumulative probability to calculate probability of nuclear war before 2050, but if interdependency of base rate was not used (i.e. 0.0127 * 26 = 0.33, which is equivalent to metaculus), shouldn’t we already use a Beta conjugation of the base rate as each year pass-by?
Good point! I wonder whether Metaculus’ community is taking this into account while thinking the annual risk is higher than the base rate of 1.27 %, such that 33 % until 2050 still makes sense. If Metaculus’ community is not taking the above into account, I should have ideally updated their probability downwards.
how do you calibrate the monetary incentive and limited time frame when weighing the 2 sets of questions for your research?
Interesting question! All else equal, I give more weight to questions which have a monetary incentive, and so would tend to rely on those from the Nuclear Risk Tournament over those from The Nuclear Risk Horizon Project. However, questions with monetary incentives are often part of tournaments with quite limited timeframes of a few years, which means extrapolating the results a few decades out may result in poor estimates.
For my case, I do not think I had available forecasts about the number of offensive nuclear detonations conditional on at least one before 2024 (or other close date). If I had, I would have to think about how much weight to give them. In the post, I compared this and this questions, but they are both part of the Nuclear Risk Horizon Project, and both have the same timeframe.
How many non-strategic nuclear weapons will be deployed at the end of 2023?
Note this question only refers to deployed nonstrategic weapons. The vast majority of deployed nuclear weapons is strategic. The US had 100 deployed nonstrategic nuclear weapons in 2023, but 1.67 k strategic.
I estimated 392 M famine deaths due to the climatic effects conditional on at least 1.07 k offensive nuclear detonations, so Metaculus’ community prediction of 45 % probability of over 4 billion deaths seems pessimistic. On the other hand, it may be the case that famine deaths due to the climatic effects are only a small fraction of the overall deaths (and Metaculus’ prediction refers to the overall deaths). For example, maybe Metaculus’ community is thinking that large nuclear wars would happen together with bio or AI great power war, thus predicting a higher death toll.
Xia 2022 presents various death rates in Fig. 5b for various levels of adaptation. Depending on how one thinks the response would go, one can deem the baseline estimates from the Rutgers’ team too optimistic/pessimistic. In order to represent the response, I used their scenario for no international food trade, no household food waste, and all livestock grain fed to humans, adjusted to include international food trade without equitable distribution. However, there is huge uncertainty, so I considered famine deaths can vary by a factor of 100 even for fixed soot injected into the stratosphere.
I am generally going on the heuristic on prediction market probably have an upper hand in counting weapons and predicting deployment and number and location of detonation, but not on long drag-out nuclear winter affects (crop yield, trade, famine numbers).
I think prediction markets often add value by estimating the likelihood of various scenarios. For example, Xia 2022 presents results for:
Various nuclear wars (from 5 Tg to 150 Tg), but does not mention the likelihood of each one of them.
The number of deaths conditional on various responses, but does not provide a best guess for the mortality corresponding to a best guess response.
I still need time to engage with the soot calculation literature, so I will probably write a follow-up on that later next week or the week after if that’s okay, that will give me much more focus on asking the right questions and doing the right research.
I understand the desire to use cumulative probability to calculate probability of nuclear war before 2050, but if interdependency of base rate was not used (i.e. 0.0127 * 26 = 0.33, which is equivalent to metaculus), shouldn’t we already use a Beta conjugation of the base rate as each year pass-by?
- If detonation does not happen, Beta(1, 79)
- If detonation happen, Beta(2, 79)
- annual probability = 0.0127
- Cumulative probability of 21.843% by 2050
I saw you use Beta distribution for the CDF constraint the probability of a large nuclear war, defined using the metaculus question, I agree with this, I think this checks-out. I also like that you give less weighting to the metaculus question that ask for probability distribution as it will be less accurate than taking the Beta distribution of 100 to 1000, I learnt something about how to evaluate metaculus question here:
There seems to be 2 set of questions regarding nuclear impact and winter:
- The Nuclear Risk Horizon Project (no monetary incentive)
- Nuclear Risk Tournament ($ 2685.5 reward, and ends on 1st Feb 2024)
I wan to understand how do you calibrate the monetary incentive and limited time frame when weighing the 2 sets of questions for your research?
For example contrasting these 2 questions, which you have addressed in your post:
How many nuclear weapons will be detonated offensively by 2050, if at least one offensive detonation occurs? [HORIZON, non-monetary]
How many non-strategic nuclear weapons will be deployed at the end of 2023? (No recency weighted)? [TOURNAMENT, monetary]
The deployment mean is an orders of magnitude higher than predicted detonation. Surely, even 100 weapons is a very contained regional war scenario according to Hochmann et al (2021). And a very constrained exchange between Russia/China and NATO. I would think that the former question and prediction unrealistically low given how many test just NK have conducted recently. I think you have adequately modelled that with your beta-distribution, but that will be 3x higher than the latter question unweighted results which is about 112 at median, and 161 weapons at the 75th percentile (11 Tg soot), and the 95th percentile of your calculation of 1.81k is 3x the latter question’s distribution, do you think there’s a need to reconcile that?.
How do you feel about taking expected value of such numbers https://www.metaculus.com/questions/8382/1000-nuke-detonations-cause-4b-deaths/ (4 Bil * 0.45) when this seems so far lower than numbers proposed by more sophisticated modelling, esp the Rutgers Team. I am generally going on the heuristic on prediction market probably have an upper hand in counting weapons and predicting deployment and number and location of detonation, but not on long drag-out nuclear winter affects (crop yield, trade, famine numbers).
I still need time to engage with the soot calculation literature, so I will probably write a follow-up on that later next week or the week after if that’s okay, that will give me much more focus on asking the right questions and doing the right research.
Thanks for looking into my post, Chris!
Good point! I wonder whether Metaculus’ community is taking this into account while thinking the annual risk is higher than the base rate of 1.27 %, such that 33 % until 2050 still makes sense. If Metaculus’ community is not taking the above into account, I should have ideally updated their probability downwards.
Interesting question! All else equal, I give more weight to questions which have a monetary incentive, and so would tend to rely on those from the Nuclear Risk Tournament over those from The Nuclear Risk Horizon Project. However, questions with monetary incentives are often part of tournaments with quite limited timeframes of a few years, which means extrapolating the results a few decades out may result in poor estimates.
For my case, I do not think I had available forecasts about the number of offensive nuclear detonations conditional on at least one before 2024 (or other close date). If I had, I would have to think about how much weight to give them. In the post, I compared this and this questions, but they are both part of the Nuclear Risk Horizon Project, and both have the same timeframe.
Note this question only refers to deployed nonstrategic weapons. The vast majority of deployed nuclear weapons is strategic. The US had 100 deployed nonstrategic nuclear weapons in 2023, but 1.67 k strategic.
I estimated 392 M famine deaths due to the climatic effects conditional on at least 1.07 k offensive nuclear detonations, so Metaculus’ community prediction of 45 % probability of over 4 billion deaths seems pessimistic. On the other hand, it may be the case that famine deaths due to the climatic effects are only a small fraction of the overall deaths (and Metaculus’ prediction refers to the overall deaths). For example, maybe Metaculus’ community is thinking that large nuclear wars would happen together with bio or AI great power war, thus predicting a higher death toll.
Xia 2022 presents various death rates in Fig. 5b for various levels of adaptation. Depending on how one thinks the response would go, one can deem the baseline estimates from the Rutgers’ team too optimistic/pessimistic. In order to represent the response, I used their scenario for no international food trade, no household food waste, and all livestock grain fed to humans, adjusted to include international food trade without equitable distribution. However, there is huge uncertainty, so I considered famine deaths can vary by a factor of 100 even for fixed soot injected into the stratosphere.
I think prediction markets often add value by estimating the likelihood of various scenarios. For example, Xia 2022 presents results for:
Various nuclear wars (from 5 Tg to 150 Tg), but does not mention the likelihood of each one of them.
The number of deaths conditional on various responses, but does not provide a best guess for the mortality corresponding to a best guess response.
Sure!