Thanks for doing this. Why is the next-month chance of nuclear war so low? (0.034%) That seems like basically what the base rate should be, i.e. the chance of an average month in an average year resulting in nuclear war. But this is clearly not an average month, the situation is clearly much more risky. How much more, I dunno. But probably at least an order of magnitude more, maybe two? Did you model this, or do we just disagree that this is a significantly riskier month than average?
So to be 100x of the default rate, one should have put less than 1% on events in Ukraine unfolding as they are now. This feels too low to me (and I registered some predictions in personal communications about a year ago supporting that — thou done in a hurry of email exchange).
I think a reasonable forecaster working on nuclear risk should have put significant worsening of the situation at above 10% (just from crude base rates of Russia’s “foreign policy” and past engagement in Ukraine). And I think among Russia-Ukrainian conflicts, this one, while surprising (to me subjectively) in a few ways, is not in the bottom 10% (and not even in the bottom third for me in terms of Russia-NATO tensions — based on the past forecasts). So one should go above baseline but no more than an order of magnitude, imo.
So mechanistically, one our forecasters was really low on that, as you can see on the min column. That’s one of the reasons why we use the aggregate without min/max as our best guess.
Separately:
We are not talking about nuclear war in general, we are talking about NATO/Russia specifically. So if Russia drops a nuclear bomb in Ukraine, that doesn’t per see resolve the relevant subquestion positively.
We probably do disagree with you on the base rate, and think it’s a bits lower than what Laplace’s law would imply, partially because Laplace tends to overestimate generically, and partially because nobody wants nuclear war.it We each did some rudimentary modeling for this.
It’s actually relatively late in the game, and it seems like Putin/NATO have figured out where the edges are (e.g., a no fly zone is not acceptable, but arming Ukranians is)
I don’t think this is a situation where the per-month base rate applies. The history of nuclear war is not a flat rate of risk every month; instead, the risk is lower most months and then much higher on some months, and it averages out to whatever flat rate you think is appropriate. (Looking at the list of nuclear close calls it seems hard to believe the overall chance of nuclear war was <50% for the last 70 years. Individual incidents like the cuban missile crisis seem to contribute at least 20%.) So there should be some analysis of how this month compares to past months—is it more like one of the high-risk months, or more like a regular month?
ETA: Analogy: Just a few hours ago I was wondering how likely it was that my flight would be cancelled or delayed. The vast majority of flights are not cancelled or delayed. However, I didn’t bother to calculate this base rate, because I have the relevant information that there is a winter storm happening right now around the airport. So instead I tried to use a narrower reference class—how often do flights get cancelled in this region during winter storms? Similarly, I think the reference class to use is something like “Major flare-up in tensions / crisis between NATO and Russia” not “average month in average year.” We should be looking at the cuban missile crisis, the fall of the soviet union, and maybe a half-dozen other examples, and using those as our reference class. Unless we think the current situation is milder than that.
I take the point no 3 above that it’s relatively late in the game, the edges seem mostly agreed-upon, etc. This is the sort of analysis that I think should be done. I would like to see a brainstorm of scenarios that could lead to war & an attempt to estimate their probability. Like, if Putin uses nukes in Ukraine, might that escalate? What if he uses chemical WMDs? What if NATO decides to ship in more warplanes and drones? What if they start going after each other’s satellites, since satellites are so militarily important? What if there is an attempted assassination or coup?
(Re point 1: Sure, yeah, but probably NATO vs. Russia risk is within an order of magnitude of general nuclear war risk, and plausibly more than 50% as high. Most of the past nuclear close calls were NATO vs. Russia, the current crisis is a NATO vs. Russia crisis, and if there is a nuclear war right now it seems especially likely to escalate to London being hit compared to e.g. pakistan vs. India in a typical year etc.)
(Looking at the list of nuclear close calls it seems hard to believe the overall chance of nuclear war was <50% for the last 70 years. Individual incidents like the cuban missile crisis seem to contribute at least 20%.)
There’s reason to think that this isn’t the best way to interpret the history of nuclear near-misses (assuming that it’s correct to say that we’re currently in a nuclear near-miss situation, and following Nuno I think the current situation is much more like e.g. the Soviet invasion of Afghanistan than the Cuban missile crisis). I made this point in an old post of mine following something Anders Sandberg said, but I think the reasoning is valid:
Robert Wiblin: So just to be clear, you’re saying there’s a lot of near misses, but that hasn’t updated you very much in favor of thinking that the risk is very high. That’s the reverse of what we expected.
Anders Sandberg: Yeah.
Robert Wiblin: Explain the reasoning there.
Anders Sandberg: So imagine a world that has a lot of nuclear warheads. So if there is a nuclear war, it’s guaranteed to wipe out humanity, and then you compare that to a world where is a few warheads. So if there’s a nuclear war, the risk is relatively small. Now in the first dangerous world, you would have a very strong deflection. Even getting close to the state of nuclear war would be strongly disfavored because most histories close to nuclear war end up with no observers left at all.
In the second one, you get the much weaker effect, and now over time you can plot when the near misses happen and the number of nuclear warheads, and you actually see that they don’t behave as strongly as you would think. If there was a very strong anthropic effect you would expect very few near misses during the height of the Cold War, and in fact you see roughly the opposite. So this is weirdly reassuring. In some sense the Petrov incident implies that we are slightly safer about nuclear war.
Essentially, since we did often get ‘close’ to a nuclear war without one breaking out, we can’t have actually been that close to nuclear annihilation, or all those near-misses would be too unlikely (both on ordinary probabilistic grounds since a nuclear war hasn’t happened, and potentially also on anthropic grounds since we still exist as observers).
Basically, this implies our appropriate base rate given that we’re in something the future would call a nuclear near-miss shouldn’t be really high.
However, I’m not sure what this reasoning has to say about the probability of a nuclear bomb being exploded in anger at all. It seems like that’s outside the reference class of events Sandberg is talking about in that quote. FWIW Metaculus has that at 10% probability.
We should be looking at the cuban missile crisis, the fall of the soviet union, and maybe a half-dozen other examples, and using those as our reference class. Unless we think the current situation is milder than that.
Yeah, we do think the situation is much lower than that. Reference class points might be the invasion of Georgia, the invasion of Crimea, the invasion of Afghanistan, the Armenia and Azerbaijan war, the invasion of Finland, etc.
Seems less mild than the invasion of Georgia and the Armenia and Azerbaijan war. But more mild than the cuban missile crisis for sure. Anyhow, roughly how many examples do you think you’d want to pick for the reference class? If it’s a dozen or so, then I think you’d get a substantially higher base rate.
Agreed. Though that was a century ago & with different governments, as you pointed out elsewhere. Also no nukes; presumably nukes make escalation less likely than it was then (I didn’t realize this until now when I just read the wiki—apparently the Allies almost declared war on the soviet union due to the invasion of finland!).
To be clear I haven’t thought about this nearly as much as you. I just think that this is clearly an unusually risky month and so the number should be substantially higher than for an average month.
I agree the risk should be substantially higher than for an average month and I think most Samotsvety forecasters agree. I think a large part of the disagreement may be on how risky the average month is.
From the post:
(a) may be due to having a lower level of baseline risk before adjusting up based on the current situation. For example, while Luisa Rodríguez’s analysis puts the chance of a US/Russia nuclear exchange at .38%/year. We think this seems too high for the post-Cold War era after new de-escalation methods have been implemented and lessons have been learnt from close calls. Additionally, we trust the superforecaster aggregate the most out of the estimates aggregated in the post.
Speaking personally, I’d put the baseline risk per year at ~.1%/yr then have adjusted up by a factor of 10 to ~1%/yr given the current situation, which gives me ~.08%/month which is pretty close to the aggregate of ~.07%.
We also looked some from alternative perspectives e.g. decomposing Putin’s decision making process, which gave estimates in the same ballpark.
I take the point no 3 above that it’s relatively late in the game, the edges seem mostly agreed-upon, etc. This is the sort of analysis that I think should be done. I would like to see a brainstorm of scenarios that could lead to war & an attempt to estimate their probability.
I agree that this would be good to have in the abstract. But it would also be much more expensive and time-consuming, and I expect not all that much better than our current guess and current more informal/intuitive/unwritten models.
And because I/we think that this is probably not a Cuban missile crisis situation, I think it would mostly not be worth it.
Fair enough! I’m grateful for all the work you’ve already done and don’t think it’s your job to do more research in the areas that would be more convincing to me.
Most of the past nuclear close calls were NATO vs. Russia, the current crisis is a NATO vs. Russia crisis
This feels wrong. Like, an important feature of the current situation is that Ukraine is in fact not part of NATO. Further, previous crises were between the Soviet Union and the Western Capitalists, which were much more ideologically opposed.
Not to toot my own horn too much, but I think that my post gives a decent explanation for why you shouldn’t update too much on recent events—I’m still higher than the consensus, but I expected to be much higher.
Not sure this is a good place to comment on your blogpost, but I found myself having very different intuitions about this paragraph:
So: what’s the probability of something like the Ukraine situation given a nuclear exchange this year? I’d actually expect a nuclear exchange to be precipitated by somewhat more direct conflict, rather than something more proxy-like. For instance, maybe we’d expect Russia to talk about how Estonia is rightfully theirs, and how it shouldn’t even be a big deal to NATO, rather than the current world where the focus has been on Ukraine specifically for a while. So I’d give this conditional probability as 1⁄3, which is about 3⁄10.
I think the game theory around NATO precommitments and deterrence more generally gets especially messy in the “proxy-like” situation, so much so that I remember reading takes where people thought escalation was unusually likely in the sort of situation we’re in. The main reason being that if the situation is messier, it’s not obvious what each side considers unacceptable, so you increase the odds of falsely modeling the other player.
It’s true that we can think of situations that would be even more dangerous than what we see now, but if those other situations were extremely unlikely a priori, then that still leaves more probability mass for things like the current scenario when you condition on “nuclear exchange this year.” Out of all the dangerous things Russia was likely to do, isn’t a large invasion into Ukraine pretty high on the list? (Maybe this point is compatible with what you write; I’m not sure I understood it exactly.)
Putin seems to have gone off the rails quite a bit and the situation in Russia for him is pretty threatening (he’s got the country hostage but has to worry about attempts to out him). Isn’t that the sort of thing you’d expect to see before an exchange? Maybe game theory is overrated and you care more about the state of mind of the dictator and the sort of conditions of the country (whether people expect their family to be imprisoned if they object to orders that lead to more escalation, etc.). So I was surprised that you didn’t mention points related to this.
Re: 1, my impression is that Russia and NATO have been in a few proxy conflicts, and so have presumably worked out this deal.
Re: 2, that seems basically right.
Re: 3, I’d weight this relatively low, since it’s the kind of ‘soft’ reporting I’d strongly expect to be less reliable in the current media environment where Putin is considered a villain.
Note that at most this will get you an order of magnitude above the base rate.
Thanks for doing this. Why is the next-month chance of nuclear war so low? (0.034%) That seems like basically what the base rate should be, i.e. the chance of an average month in an average year resulting in nuclear war. But this is clearly not an average month, the situation is clearly much more risky. How much more, I dunno. But probably at least an order of magnitude more, maybe two? Did you model this, or do we just disagree that this is a significantly riskier month than average?
So to be 100x of the default rate, one should have put less than 1% on events in Ukraine unfolding as they are now. This feels too low to me (and I registered some predictions in personal communications about a year ago supporting that — thou done in a hurry of email exchange).
I think a reasonable forecaster working on nuclear risk should have put significant worsening of the situation at above 10% (just from crude base rates of Russia’s “foreign policy” and past engagement in Ukraine). And I think among Russia-Ukrainian conflicts, this one, while surprising (to me subjectively) in a few ways, is not in the bottom 10% (and not even in the bottom third for me in terms of Russia-NATO tensions — based on the past forecasts). So one should go above baseline but no more than an order of magnitude, imo.
Ah, that’s a good argument, thanks! Updating downwards.
So mechanistically, one our forecasters was really low on that, as you can see on the min column. That’s one of the reasons why we use the aggregate without min/max as our best guess.
Separately:
We are not talking about nuclear war in general, we are talking about NATO/Russia specifically. So if Russia drops a nuclear bomb in Ukraine, that doesn’t per see resolve the relevant subquestion positively.
We probably do disagree with you on the base rate, and think it’s a bits lower than what Laplace’s law would imply, partially because Laplace tends to overestimate generically, and partially because nobody wants nuclear war.it We each did some rudimentary modeling for this.
It’s actually relatively late in the game, and it seems like Putin/NATO have figured out where the edges are (e.g., a no fly zone is not acceptable, but arming Ukranians is)
I don’t think this is a situation where the per-month base rate applies. The history of nuclear war is not a flat rate of risk every month; instead, the risk is lower most months and then much higher on some months, and it averages out to whatever flat rate you think is appropriate. (Looking at the list of nuclear close calls it seems hard to believe the overall chance of nuclear war was <50% for the last 70 years. Individual incidents like the cuban missile crisis seem to contribute at least 20%.) So there should be some analysis of how this month compares to past months—is it more like one of the high-risk months, or more like a regular month?
ETA: Analogy: Just a few hours ago I was wondering how likely it was that my flight would be cancelled or delayed. The vast majority of flights are not cancelled or delayed. However, I didn’t bother to calculate this base rate, because I have the relevant information that there is a winter storm happening right now around the airport. So instead I tried to use a narrower reference class—how often do flights get cancelled in this region during winter storms? Similarly, I think the reference class to use is something like “Major flare-up in tensions / crisis between NATO and Russia” not “average month in average year.” We should be looking at the cuban missile crisis, the fall of the soviet union, and maybe a half-dozen other examples, and using those as our reference class. Unless we think the current situation is milder than that.
I take the point no 3 above that it’s relatively late in the game, the edges seem mostly agreed-upon, etc. This is the sort of analysis that I think should be done. I would like to see a brainstorm of scenarios that could lead to war & an attempt to estimate their probability. Like, if Putin uses nukes in Ukraine, might that escalate? What if he uses chemical WMDs? What if NATO decides to ship in more warplanes and drones? What if they start going after each other’s satellites, since satellites are so militarily important? What if there is an attempted assassination or coup?
(Re point 1: Sure, yeah, but probably NATO vs. Russia risk is within an order of magnitude of general nuclear war risk, and plausibly more than 50% as high. Most of the past nuclear close calls were NATO vs. Russia, the current crisis is a NATO vs. Russia crisis, and if there is a nuclear war right now it seems especially likely to escalate to London being hit compared to e.g. pakistan vs. India in a typical year etc.)
There’s reason to think that this isn’t the best way to interpret the history of nuclear near-misses (assuming that it’s correct to say that we’re currently in a nuclear near-miss situation, and following Nuno I think the current situation is much more like e.g. the Soviet invasion of Afghanistan than the Cuban missile crisis). I made this point in an old post of mine following something Anders Sandberg said, but I think the reasoning is valid:
Essentially, since we did often get ‘close’ to a nuclear war without one breaking out, we can’t have actually been that close to nuclear annihilation, or all those near-misses would be too unlikely (both on ordinary probabilistic grounds since a nuclear war hasn’t happened, and potentially also on anthropic grounds since we still exist as observers).
Basically, this implies our appropriate base rate given that we’re in something the future would call a nuclear near-miss shouldn’t be really high.
However, I’m not sure what this reasoning has to say about the probability of a nuclear bomb being exploded in anger at all. It seems like that’s outside the reference class of events Sandberg is talking about in that quote. FWIW Metaculus has that at 10% probability.
What is the appropriate reference class:
Yeah, we do think the situation is much lower than that. Reference class points might be the invasion of Georgia, the invasion of Crimea, the invasion of Afghanistan, the Armenia and Azerbaijan war, the invasion of Finland, etc.
Seems less mild than the invasion of Georgia and the Armenia and Azerbaijan war. But more mild than the cuban missile crisis for sure. Anyhow, roughly how many examples do you think you’d want to pick for the reference class? If it’s a dozen or so, then I think you’d get a substantially higher base rate.
(As an aside, I think that the invasion of Finland is actually a great reference point)
Agreed. Though that was a century ago & with different governments, as you pointed out elsewhere. Also no nukes; presumably nukes make escalation less likely than it was then (I didn’t realize this until now when I just read the wiki—apparently the Allies almost declared war on the soviet union due to the invasion of finland!).
To be clear I haven’t thought about this nearly as much as you. I just think that this is clearly an unusually risky month and so the number should be substantially higher than for an average month.
I agree the risk should be substantially higher than for an average month and I think most Samotsvety forecasters agree. I think a large part of the disagreement may be on how risky the average month is.
From the post:
Speaking personally, I’d put the baseline risk per year at ~.1%/yr then have adjusted up by a factor of 10 to ~1%/yr given the current situation, which gives me ~.08%/month which is pretty close to the aggregate of ~.07%.
We also looked some from alternative perspectives e.g. decomposing Putin’s decision making process, which gave estimates in the same ballpark.
OK, thanks!
Deeper analysis needed?
I agree that this would be good to have in the abstract. But it would also be much more expensive and time-consuming, and I expect not all that much better than our current guess and current more informal/intuitive/unwritten models.
And because I/we think that this is probably not a Cuban missile crisis situation, I think it would mostly not be worth it.
Fair enough! I’m grateful for all the work you’ve already done and don’t think it’s your job to do more research in the areas that would be more convincing to me.
NATO : Russia :: NATO : Soviet Union?
This feels wrong. Like, an important feature of the current situation is that Ukraine is in fact not part of NATO. Further, previous crises were between the Soviet Union and the Western Capitalists, which were much more ideologically opposed.
Not to toot my own horn too much, but I think that my post gives a decent explanation for why you shouldn’t update too much on recent events—I’m still higher than the consensus, but I expected to be much higher.
Not sure this is a good place to comment on your blogpost, but I found myself having very different intuitions about this paragraph:
I think the game theory around NATO precommitments and deterrence more generally gets especially messy in the “proxy-like” situation, so much so that I remember reading takes where people thought escalation was unusually likely in the sort of situation we’re in. The main reason being that if the situation is messier, it’s not obvious what each side considers unacceptable, so you increase the odds of falsely modeling the other player.
It’s true that we can think of situations that would be even more dangerous than what we see now, but if those other situations were extremely unlikely a priori, then that still leaves more probability mass for things like the current scenario when you condition on “nuclear exchange this year.” Out of all the dangerous things Russia was likely to do, isn’t a large invasion into Ukraine pretty high on the list? (Maybe this point is compatible with what you write; I’m not sure I understood it exactly.)
Putin seems to have gone off the rails quite a bit and the situation in Russia for him is pretty threatening (he’s got the country hostage but has to worry about attempts to out him). Isn’t that the sort of thing you’d expect to see before an exchange? Maybe game theory is overrated and you care more about the state of mind of the dictator and the sort of conditions of the country (whether people expect their family to be imprisoned if they object to orders that lead to more escalation, etc.). So I was surprised that you didn’t mention points related to this.
Re: 1, my impression is that Russia and NATO have been in a few proxy conflicts, and so have presumably worked out this deal.
Re: 2, that seems basically right.
Re: 3, I’d weight this relatively low, since it’s the kind of ‘soft’ reporting I’d strongly expect to be less reliable in the current media environment where Putin is considered a villain.
Note that at most this will get you an order of magnitude above the base rate.
Oh nice, thanks, this is the sort of thing I was looking for!