Thanks for going through the “premises” and leaving your comments on each—very helpful for myself to further clarify and reflect upon my thoughts!
On P1 (that nuclear escalation is the main or only path to existential catastrophe):
Yes, I do argue for the larger claim that a one-time deployment of nuclear weapons could be the start of a development that ends in existential catastrophe even if there is no nuclear escalation.
I give a partial justification of that in the post and in my comment to Aron,
but I accept that it’s not completely illegitimate for people to continue to disagree with me; opinions on a question like this rest on quite foundational beliefs, intuitions, and heuristics, and two reasonable people can, imo, have different sets of these.
(Would love to get into a more in-depth conversation on this question at some point though, so I’d suggest putting it on the agenda for the next time we happen to see each other in-person :)!)
On P2:
Your suggested reformulation (“preventing the first nuclear deployment is more tractable because preventing escalation has more unknowns”) is pretty much in line with what I meant this premise/proposition to say in the context of my overall argument. So, on a high-level, this doesn’t seem like a crux that would lead the two of us to take a differing stance on my overall conclusion.
You’re right that I’m not very enthusiastic about the idea of putting actual probabilities on any of the event categories I mention in the post (event categories: possible consequences of a one-time deployment of nukes; conceivable effects of different types of interventions). We’re not even close to sure that we/I have succeeded in identifying the range of possible consequences (pathways to existential catastrophe) and effects (of interventions), and those consequences and effects that I did identify aren’t very specific or well-defined; both of these seem like necessary prudent steps to precede the assignment of probabilities. I realize while writing that you will probably just once again disagree with that leap I made (from deep uncertainty to rejecting probability assignment), and that I’m not doing much to advance our discussion here. Apologies! On your specific points: correct, I don’t think we can advance much beyond an intuitive, extremely uncertain assignment of probabilities; I think that the alternative (whose existence you deny) is to acknowledge our lack of reasonable certainty about these probabilities and to make decisions in the awareness that there are these unknowns (in our model of the world); and I (unsurprisingly) disagree that institutions or people that choose this alternative will do systematically worse than those that always assign probabilities.
(I don’t think the start-up analogy is a good one in this context, since venture capitalists get to make many bets and they receive reliable and repeated feedback on their bets. Neither of these seem particularly true in the nuclear risk field (whether we’re talking about assigning probabilities to the consequences of nuclear weapons deployment or about the effects of interventions to reduce escalation risk / prepare for a post-nuclear war world).)
On P3: Thanks for flagging that, even after reading my post, you feel ill-equipped to assess my claim regarding the value of interventions for preventing first-use vs. interventions for preventing further escalation. Enabling readers to navigate, understand and form an opinion on claims like that one was one of the core goals that I started this summer’s research fellowship with; I shall reflect on whether this post could have been different, or whether there could have been a complementary post, to better achieve this enabling function!
On P4: Haha yes, I see this now, thanks for pointing it out! I’m wondering whether renaming them “propositions” or “claims” would be more appropriate?
Fair enough re the disanalogy between investing and nuclear predictions. My strong suspicion (from talking to some of them + reading write-ups + vibes) is that the people with the best Brier scores in various forecasting competitions use probabilities plenty, and probably more than the mediocre forecasters who just intuit a final number. But then I think you could reasonably respond that this is a selection effect that of course the people who love probabilities will spend lots of time forecasting and rise to the top.
I wonder what adversarial collaboration would resolve this probabilities issue? Perhaps something like: get a random group of a few hundred people and assign one group some calibration training and probability theory and practice and so forth a bit like what Nuno gave us, and assign the other group a curriculum of your choice that doesn’t emphasise probabilities. Then get them all to make a bunch of forecasts, and a year or however long later see which group did better. This already seems skewed in my favour though as the forecasts are just probabilities. I am struggling to think of a way to test predictive skill nicely without using probability forecasts though.
Re your goals for the summer project, oh no that certainly wasn’t the vibe I was intending to convey. I think this post succeeded in making me understand some of the issues at stake here better, and get a sense for some of the arguments and tensions. I think if after reading a 3000-word article with minimal background knowledge I had come to form strong and well-founded inside takes on (as you say) a very complex issue like nuclear policy, this would be remarkable! I don’t think it is a failing of you or the essay at all that this isn’t (very much) the case. To have good inside views I think I would need to at least be familiar with the various nuclear research/policy agendas out there, and how they fit into the preventing first use/preventing 100-->1000 escalation, and how tractable they each seem.
I think I still like calling them each ‘arguments’ but I think anything is probably better than ‘premise’ given the very special meaning of that word in logic.
Ah, I think mybe there is/was a misunderstanding here. I don’t reject the claim that the forecasters are (much) better on average when using probabilities than when refusing to do so. I think my point here is that the questions we’re talking about (what would be the full set of important* consequences of nuclear first-use or the full set of important* consequences of nuclear risk reduction interventions X and Z) are not your standard, well-defined and soon-to-be-resolved forecasting questions. So in a sense, the very fact that the questions at issue cannot be part of a forecasting experiment is one of the main reasons for why I think they are so deeply uncertain and hard to answer with more than intuitive guesswork (if they could be part of a forecasting experiment, people could test and train their skills at assigning probabilities by answering many such questions, in which case I guess I would be more amenable to the claim that assigning probabilities can be useful). The way I understood our disagreement, it was not about the predictive performance of actors who do vs don’t (always) use probabilities, but rather about their decision quality. I think the actual disagreement may be that I think that there is a significant difference (for some decisions, high decision quality is not a neat function of explicit predictive ability), whereas you might be close to equating the two?
[*by “full set” I mean that this is supposed to include indirect/second-order consequences]
That said, I can’t, unfortunately, think of any alternative ways to resolve the disagreement regarding the decision quality of people using vs. refusing to use probabilities in situations where assessing the effects of a decision/action after the fact is highly difficult… (While the comment added by Noah Scales contains some interesting ideas, I don’t think it does anything to resolve this stalemate, since it is also focused on comparing & assessing predictive success for questions with a small set of known answer options)
One other thing, because I forgot about that in my last response:
“FInally, I am not that sure of your internal history but one worry would be if you decided long ago intuitively based on the cultural milieu that the right answer is ‘the best intervention in nuclear policy is to try to prevent first use’ and then subconsciously sought out supporting arguments. I am not saying this is what happened or that you are any more guilty of this than me or anyone else, just that it is something I and we all should be wary of.”
-> I think this is a super important point, actually, and agree that it’s a concern that should be kept in mind when reading my essay on this topic. I did have the intuitive aversion against focusing on tail end risks before I came up with all the supporting arguments; basically, this post came about as a result of me asking myself “Why do I think it’s such a horrible idea to focus on the prevention of and preparation for the worst case of a nuclear confrontation?” I added a footnote to be more transparent about this towards the beginning of the post (fn. 2). Thanks for raising it!
(While the comment added by Noah Scales contains some interesting ideas, I don’t think it does anything to resolve this stalemate, since it is also focused on comparing & assessing predictive success for questions with a small set of known answer options)
Yes, that’s right that my suggestions let you assess predictive success, in some cases, for example, over a set of futures partitioning a space of possibilities. Since the futures partition the space, one of them will occur, the rest will not. A yes/no forecast works this way.
Actually, if you have any question about a future at a specific time about which you feel uncertain, you can phrase it as a yes/no question. You then partition the space of possibilities at that future time. Now you can answer the question, and test your predictive success. Whether your answer has any value is the concern.
However, one option I mentioned is to list contingencies that, if present, result in contingent situations (futures). That is not the same as predicting the future, since the contingencies don’t have to be present or identified (EDIT: in the real world, ie as facts), and you do not expect their contingent futures otherwise.
If condition X, then important future Y happens.
Condition X could be present now or later, but I can’t identify or infer its presence now.
Deep uncertainty is usually taken as responding to those contingent situations as meaningful anyway. As someone without predictive information, you can only start offering models, like:
If X, then Y
If Y, then Z
If W and G, then B
If B, then C
A
T
I’m worried that A because …
You can talk about scenarios, but you don’t know or haven’t seen their predictive indicators.
You can discuss contingent situations, but you can’t claim that they will occur.
You can still work to prevent those contingent situations, and that seems to be your intention in your area of research. For example, you can work to prevent current condition “A”, whatever that is. Nuclear proliferation, maybe, or deployment of battlefield nukes. Nice!
You are not asking the question, “What will the future be?” without any idea of what some scenarios of the future depend on. After all, if the future is a nuclear holocaust, you can backtrack to at least some earlier point in time, for example, far enough to determine that nuclear weapons were detonated prior to the holocaust, and further to someone or something detonating them, and then maybe further to who had them, or why they detonated them, or that might be where gaps in knowledge appear.
Yes, I think this captures our difference in views pretty well: I do indeed think predictive accuracy is very valuable for decision quality. Of course, there are other skills/attributes that are also useful for making good decisions. Predicting the future seems pretty key though.
″ I am struggling to think of a way to test predictive skill nicely without using probability forecasts though.”
The list of forecasting methods that do not rely on subjective or unverifiable probability estimates includes:
in response to a yes/no forecasting question, given what you believe now about the current situation and what it will cause, answer with whether the questioned future scenario is what the current situation leads to.
in response to a multiple option (mutually exclusive choice) forecasting question, given what you believe now about the current situation and what it could cause, answer what alternative future scenarios the current situation could lead to.
in response to any forecasting question, if you lack sufficient knowledge of the facts of the current situation, then list the contingencies and contingent situations that you would match to better knowledge of the facts in order to determine the future.
in response to any forecasting question, if you lack sufficient knowledge of causal relations that lead to a scenario, backtrack necessary (and together sufficient) causes from the scenario (one of the alternative forecast answers) to as close to the present as you can. From there, decide whether the causal gap between the present situation and that near-future situation (backtracked from the forecast scenario) is plausible to causally bridge. In other words, is that near-future situation causally possible? If you don’t believe so, reject the scenario in your forecast. If you do believe so, you have a starting point for research to close the information gap corresponding to the causal gap.
The list of forecasting methods that do not provide a final forecast probability includes:
for a yes/no forecast, use whatever probability methods to reach a conclusion however you do. Then establish a probability floor below which the answer is “no”, a ceiling above which the answer is “yes”, and a range between the floor and ceiling an answer of “maybe”. Work to reduce your uncertainty enough to move your forecast probability below or above the maybe zone.
for an alternatives forecast,use whatever probability methods to reach a conclusion however you do. Then establish a floor of probability above which an option is selected. List all options above the floor as chosen alternatives (for example, “I forecast scenario A or B but not C or D.”). Work to reduce your uncertainty enough to concentrate forecast probability within fewer options (for example, “I forecast scenario B only, not A or C or D.”).
The definition of scenario in the domain of forecasting (as I use it here) includes:
an option among several that answer a forecast multiple-choice (single-answer) question.
a proposed future about which a yes/no forecast is made.
Thanks for going through the “premises” and leaving your comments on each—very helpful for myself to further clarify and reflect upon my thoughts!
On P1 (that nuclear escalation is the main or only path to existential catastrophe):
Yes, I do argue for the larger claim that a one-time deployment of nuclear weapons could be the start of a development that ends in existential catastrophe even if there is no nuclear escalation.
I give a partial justification of that in the post and in my comment to Aron,
but I accept that it’s not completely illegitimate for people to continue to disagree with me; opinions on a question like this rest on quite foundational beliefs, intuitions, and heuristics, and two reasonable people can, imo, have different sets of these.
(Would love to get into a more in-depth conversation on this question at some point though, so I’d suggest putting it on the agenda for the next time we happen to see each other in-person :)!)
On P2:
Your suggested reformulation (“preventing the first nuclear deployment is more tractable because preventing escalation has more unknowns”) is pretty much in line with what I meant this premise/proposition to say in the context of my overall argument. So, on a high-level, this doesn’t seem like a crux that would lead the two of us to take a differing stance on my overall conclusion.
You’re right that I’m not very enthusiastic about the idea of putting actual probabilities on any of the event categories I mention in the post (event categories: possible consequences of a one-time deployment of nukes; conceivable effects of different types of interventions). We’re not even close to sure that we/I have succeeded in identifying the range of possible consequences (pathways to existential catastrophe) and effects (of interventions), and those consequences and effects that I did identify aren’t very specific or well-defined; both of these seem like
necessaryprudent steps to precede the assignment of probabilities. I realize while writing that you will probably just once again disagree with that leap I made (from deep uncertainty to rejecting probability assignment), and that I’m not doing much to advance our discussion here. Apologies! On your specific points: correct, I don’t think we can advance much beyond an intuitive, extremely uncertain assignment of probabilities; I think that the alternative (whose existence you deny) is to acknowledge our lack of reasonable certainty about these probabilities and to make decisions in the awareness that there are these unknowns (in our model of the world); and I (unsurprisingly) disagree that institutions or people that choose this alternative will do systematically worse than those that always assign probabilities.(I don’t think the start-up analogy is a good one in this context, since venture capitalists get to make many bets and they receive reliable and repeated feedback on their bets. Neither of these seem particularly true in the nuclear risk field (whether we’re talking about assigning probabilities to the consequences of nuclear weapons deployment or about the effects of interventions to reduce escalation risk / prepare for a post-nuclear war world).)
On P3: Thanks for flagging that, even after reading my post, you feel ill-equipped to assess my claim regarding the value of interventions for preventing first-use vs. interventions for preventing further escalation. Enabling readers to navigate, understand and form an opinion on claims like that one was one of the core goals that I started this summer’s research fellowship with; I shall reflect on whether this post could have been different, or whether there could have been a complementary post, to better achieve this enabling function!
On P4: Haha yes, I see this now, thanks for pointing it out! I’m wondering whether renaming them “propositions” or “claims” would be more appropriate?
Thanks!
Fair enough re the disanalogy between investing and nuclear predictions. My strong suspicion (from talking to some of them + reading write-ups + vibes) is that the people with the best Brier scores in various forecasting competitions use probabilities plenty, and probably more than the mediocre forecasters who just intuit a final number. But then I think you could reasonably respond that this is a selection effect that of course the people who love probabilities will spend lots of time forecasting and rise to the top.
I wonder what adversarial collaboration would resolve this probabilities issue? Perhaps something like: get a random group of a few hundred people and assign one group some calibration training and probability theory and practice and so forth a bit like what Nuno gave us, and assign the other group a curriculum of your choice that doesn’t emphasise probabilities. Then get them all to make a bunch of forecasts, and a year or however long later see which group did better. This already seems skewed in my favour though as the forecasts are just probabilities. I am struggling to think of a way to test predictive skill nicely without using probability forecasts though.
Re your goals for the summer project, oh no that certainly wasn’t the vibe I was intending to convey. I think this post succeeded in making me understand some of the issues at stake here better, and get a sense for some of the arguments and tensions. I think if after reading a 3000-word article with minimal background knowledge I had come to form strong and well-founded inside takes on (as you say) a very complex issue like nuclear policy, this would be remarkable! I don’t think it is a failing of you or the essay at all that this isn’t (very much) the case. To have good inside views I think I would need to at least be familiar with the various nuclear research/policy agendas out there, and how they fit into the preventing first use/preventing 100-->1000 escalation, and how tractable they each seem.
I think I still like calling them each ‘arguments’ but I think anything is probably better than ‘premise’ given the very special meaning of that word in logic.
Ah, I think mybe there is/was a misunderstanding here. I don’t reject the claim that the forecasters are (much) better on average when using probabilities than when refusing to do so. I think my point here is that the questions we’re talking about (what would be the full set of important* consequences of nuclear first-use or the full set of important* consequences of nuclear risk reduction interventions X and Z) are not your standard, well-defined and soon-to-be-resolved forecasting questions. So in a sense, the very fact that the questions at issue cannot be part of a forecasting experiment is one of the main reasons for why I think they are so deeply uncertain and hard to answer with more than intuitive guesswork (if they could be part of a forecasting experiment, people could test and train their skills at assigning probabilities by answering many such questions, in which case I guess I would be more amenable to the claim that assigning probabilities can be useful). The way I understood our disagreement, it was not about the predictive performance of actors who do vs don’t (always) use probabilities, but rather about their decision quality. I think the actual disagreement may be that I think that there is a significant difference (for some decisions, high decision quality is not a neat function of explicit predictive ability), whereas you might be close to equating the two?
[*by “full set” I mean that this is supposed to include indirect/second-order consequences]
That said, I can’t, unfortunately, think of any alternative ways to resolve the disagreement regarding the decision quality of people using vs. refusing to use probabilities in situations where assessing the effects of a decision/action after the fact is highly difficult… (While the comment added by Noah Scales contains some interesting ideas, I don’t think it does anything to resolve this stalemate, since it is also focused on comparing & assessing predictive success for questions with a small set of known answer options)
One other thing, because I forgot about that in my last response:
“FInally, I am not that sure of your internal history but one worry would be if you decided long ago intuitively based on the cultural milieu that the right answer is ‘the best intervention in nuclear policy is to try to prevent first use’ and then subconsciously sought out supporting arguments. I am not saying this is what happened or that you are any more guilty of this than me or anyone else, just that it is something I and we all should be wary of.”
-> I think this is a super important point, actually, and agree that it’s a concern that should be kept in mind when reading my essay on this topic. I did have the intuitive aversion against focusing on tail end risks before I came up with all the supporting arguments; basically, this post came about as a result of me asking myself “Why do I think it’s such a horrible idea to focus on the prevention of and preparation for the worst case of a nuclear confrontation?” I added a footnote to be more transparent about this towards the beginning of the post (fn. 2). Thanks for raising it!
Sarah, you wrote:
Yes, that’s right that my suggestions let you assess predictive success, in some cases, for example, over a set of futures partitioning a space of possibilities. Since the futures partition the space, one of them will occur, the rest will not. A yes/no forecast works this way.
Actually, if you have any question about a future at a specific time about which you feel uncertain, you can phrase it as a yes/no question. You then partition the space of possibilities at that future time. Now you can answer the question, and test your predictive success. Whether your answer has any value is the concern.
However, one option I mentioned is to list contingencies that, if present, result in contingent situations (futures). That is not the same as predicting the future, since the contingencies don’t have to be present or identified (EDIT: in the real world, ie as facts), and you do not expect their contingent futures otherwise.
Deep uncertainty is usually taken as responding to those contingent situations as meaningful anyway. As someone without predictive information, you can only start offering models, like:
You can talk about scenarios, but you don’t know or haven’t seen their predictive indicators.
You can discuss contingent situations, but you can’t claim that they will occur.
You can still work to prevent those contingent situations, and that seems to be your intention in your area of research. For example, you can work to prevent current condition “A”, whatever that is. Nuclear proliferation, maybe, or deployment of battlefield nukes. Nice!
You are not asking the question, “What will the future be?” without any idea of what some scenarios of the future depend on. After all, if the future is a nuclear holocaust, you can backtrack to at least some earlier point in time, for example, far enough to determine that nuclear weapons were detonated prior to the holocaust, and further to someone or something detonating them, and then maybe further to who had them, or why they detonated them, or that might be where gaps in knowledge appear.
Yes, I think this captures our difference in views pretty well: I do indeed think predictive accuracy is very valuable for decision quality. Of course, there are other skills/attributes that are also useful for making good decisions. Predicting the future seems pretty key though.
You wrote:
The list of forecasting methods that do not rely on subjective or unverifiable probability estimates includes:
in response to a yes/no forecasting question, given what you believe now about the current situation and what it will cause, answer with whether the questioned future scenario is what the current situation leads to.
in response to a multiple option (mutually exclusive choice) forecasting question, given what you believe now about the current situation and what it could cause, answer what alternative future scenarios the current situation could lead to.
in response to any forecasting question, if you lack sufficient knowledge of the facts of the current situation, then list the contingencies and contingent situations that you would match to better knowledge of the facts in order to determine the future.
in response to any forecasting question, if you lack sufficient knowledge of causal relations that lead to a scenario, backtrack necessary (and together sufficient) causes from the scenario (one of the alternative forecast answers) to as close to the present as you can. From there, decide whether the causal gap between the present situation and that near-future situation (backtracked from the forecast scenario) is plausible to causally bridge. In other words, is that near-future situation causally possible? If you don’t believe so, reject the scenario in your forecast. If you do believe so, you have a starting point for research to close the information gap corresponding to the causal gap.
The list of forecasting methods that do not provide a final forecast probability includes:
for a yes/no forecast, use whatever probability methods to reach a conclusion however you do. Then establish a probability floor below which the answer is “no”, a ceiling above which the answer is “yes”, and a range between the floor and ceiling an answer of “maybe”. Work to reduce your uncertainty enough to move your forecast probability below or above the maybe zone.
for an alternatives forecast,use whatever probability methods to reach a conclusion however you do. Then establish a floor of probability above which an option is selected. List all options above the floor as chosen alternatives (for example, “I forecast scenario A or B but not C or D.”). Work to reduce your uncertainty enough to concentrate forecast probability within fewer options (for example, “I forecast scenario B only, not A or C or D.”).
The definition of scenario in the domain of forecasting (as I use it here) includes:
an option among several that answer a forecast multiple-choice (single-answer) question.
a proposed future about which a yes/no forecast is made.