The estimate being too low by 1-2 orders of magnitude seems plausible to me independently (e.g. see the wide distribution in my Squiggle model [1]), but my confidence in the estimate is increased by it being the aggregated of several excellent forecasters, who were reasoning independently to some extent. Given that, my all-things-considered view is that 1 order of magnitude off[2] feels plausible but not likely (~25%?), and 2 orders of magnitude seems very unlikely (~5%?).
EDIT: actually looking closer at my Squiggle model I think it should be more uncertain on the first variable, something like russiaNatoNuclearexchangeInNextMonth=.0001 to .01 rather than .0001 to .003
Compared to a reference of what’s possible given the information we have, with e.g. a group of 100 excellent forecasters would get if spending 1000 hours each.
I will just note that 10x with 20% (= 25% − 5%) and 100x with 5% would/should dominate EV of your estimate. P = .75 * X + .20 * 10 * X + .05 * 100 * X = .75 X + 7 X = 7.75 X.
Great point. Perhaps we should have ideally reported the mean of this type of distribution, rather than our best guess percentages. I’m curious if you think I’m underconfident here?
Edit: Yeah I think I was underconfident, would now be at ~10% and ~0.5% for being 1 and 2 orders of magnitude too low respectively, based primarily on considerations Misha describes in another comment placing soft bounds on how much one should update from the base rate. So my estimate should still increase but not by as much (probably by about 2x, taking into account possibility of being wrong on other side as well).
That makes sense. 2 OOMs is clearly too high now that you mention it. But I stand by my 1 OOM claim though, until people convince me that really this is much more like an ordinary business-as-usual month than I currently think it is. Which could totally happen! I am not by any means an expert on this stuff, this is just my hot take!
FWIW I’d wildly guess this analysis underestimates by 1-2 orders of magnitude, see this comment thread.
A 2 order of magnitude would be a 6.7% chance of a nuclear exchange between NATO and Russia in the next month, and potentially more over the next year. This seems implausible enough to us that we would be willing to bet[1] our $15k against your $1k[2] on this, over the next month.
One might think that one can’t bet on a catastrophic event (or, in this case, a nuclear exchange between NATO and Russia in the next month). But in fact one can: the party that doesn’t believe in the catastrophic event mails the money to the party which does, which spends it now that money is worth more.
If the catastrophic event happens, the one who was right about this doesn’t have to repay. If the catastrophic event doesn’t happen, the party which was right is repaid with interest.
You’d think that this was equivalent to just getting a loan, and in fact has worse conditions. But this doesn’t disprove that this bet would have positive expected value, it only points out that the loan is even better, and you should get both.
This is on the higher end of your proposed range. But it is also worse than what we could get just exploiting prediction market inefficiencies. But we think it’s important that people could put in their money where their mouth is.
FWIW I’d wildly guess this analysis underestimates by 1-2 orders of magnitude, see this comment thread. ETA: People (e.g. Misha) have convinced me that 2 is way too high, but I still think 1 is reasonable. ETA: This forecaster has a much more in-depth analysis that says it’s 1 OOM.
The estimate being too low by 1-2 orders of magnitude seems plausible to me independently (e.g. see the wide distribution in my Squiggle model [1]), but my confidence in the estimate is increased by it being the aggregated of several excellent forecasters, who were reasoning independently to some extent. Given that, my all-things-considered view is that 1 order of magnitude off[2] feels plausible but not likely (~25%?), and 2 orders of magnitude seems very unlikely (~5%?).
EDIT: actually looking closer at my Squiggle model I think it should be more uncertain on the first variable, something like russiaNatoNuclearexchangeInNextMonth=.0001 to .01 rather than .0001 to .003
Compared to a reference of what’s possible given the information we have, with e.g. a group of 100 excellent forecasters would get if spending 1000 hours each.
I will just note that 10x with 20% (= 25% − 5%) and 100x with 5% would/should dominate EV of your estimate. P = .75 * X + .20 * 10 * X + .05 * 100 * X = .75 X + 7 X = 7.75 X.
Great point. Perhaps we should have ideally reported the mean of this type of distribution, rather than our best guess percentages. I’m curious if you think I’m underconfident here?
Edit: Yeah I think I was underconfident, would now be at ~10% and ~0.5% for being 1 and 2 orders of magnitude too low respectively, based primarily on considerations Misha describes in another comment placing soft bounds on how much one should update from the base rate. So my estimate should still increase but not by as much (probably by about 2x, taking into account possibility of being wrong on other side as well).
That makes sense. 2 OOMs is clearly too high now that you mention it. But I stand by my 1 OOM claim though, until people convince me that really this is much more like an ordinary business-as-usual month than I currently think it is. Which could totally happen! I am not by any means an expert on this stuff, this is just my hot take!
A 2 order of magnitude would be a 6.7% chance of a nuclear exchange between NATO and Russia in the next month, and potentially more over the next year. This seems implausible enough to us that we would be willing to bet[1] our $15k against your $1k[2] on this, over the next month.
One might think that one can’t bet on a catastrophic event (or, in this case, a nuclear exchange between NATO and Russia in the next month). But in fact one can: the party that doesn’t believe in the catastrophic event mails the money to the party which does, which spends it now that money is worth more.
If the catastrophic event happens, the one who was right about this doesn’t have to repay. If the catastrophic event doesn’t happen, the party which was right is repaid with interest.
You’d think that this was equivalent to just getting a loan, and in fact has worse conditions. But this doesn’t disprove that this bet would have positive expected value, it only points out that the loan is even better, and you should get both.
This is on the higher end of your proposed range. But it is also worse than what we could get just exploiting prediction market inefficiencies. But we think it’s important that people could put in their money where their mouth is.
That’s why it was my upper bound. I too think it’s pretty implausible. How would you feel about a bet on the +1 OOM odds?