Well, my basic opinion about forecasting is that probabilities don’t inform the person receiving the forecast. Before you commit to weighting possible outcomes, you commit to at least two mutually exclusive futures, X and not X. So what you supply is a limitation on possible outcomes, either X or not X. At best, you’re aware of mutually exclusive alternative and specific futures. Then you can limit what not X means to something specific, for example, Y. So now you can say, “The future will contain X or Y.” That sort of analysis is enabled by your causal model. As your causal model improves, it becomes easier to supply a list of alternative future outcomes.
However, the future is not a game of chance, and there’s no useful interpretation to supply meaningful weights to the future prediction of any specific outcome, unless the outcomes belong to a game of chance, where you’re predicting rolls of a fair die, choice of a hand from a deck of cards, etc.
What’s worse, that does not limit your feelings about what probabilities apply. Those feelings can seem real and meaningful because they let you talk about lists of outcomes and which you think are more credible.
As a forecaster, I might supply outcomes in a forecast that I consider less credible along with those that I consider more credible. but if you ask me which options I consider credible, I might offer a subset of the list. So in that way weights can seem valuable, because they let you distinguish which you think are more credible and which you can rule out. But the weights also obscure that information because they can scale that credibility in confusing ways.
For example, I believe in outcomes A or B, but I offer A at 30%, B at 30%, C at 20%, D at 10%, and E at 10%. Have I communicated what I intended with my weights, namely, that A and B are credible, that C is somewhat credible, but D and E are not? Maybe I could adjust A and B to 40% and 40%, but now I’m fiddling with the likelihoods of C, D, and E, when all I really mean to communicate is that I like A or B as outcomes and C as an alternate. My probabilities communicate more and differently than I intend. I could make it clear with A and B each at 48% or something, but really now I’m trying to pretend I know what the chances of C, D, and E are, when all I really know about them is that my causal model doesn’t support their production much. I could go back and quantify that somehow, but information with which to do that is not available , so I have to pretend confidence in some estimation of the outcomes C, D, and E. My information is not useless, but it’s not relevant to weighting all possible outcomes against each other. If I’m forced to provide weights for all the listed outcomes, then I’m forced to figure out how to communicate my analysis in terms of weights so that the audience for my forecast understands what I intend to mean.
In general, analyzing causal models that determine possible futures is a distinct activity from weighting those futures. The valuable information is in the causal models and in the selection of futures based on those models. The extra information on epistemic confidence is not useful and pretends more information than a forecaster likely has. I would go as far as two tiers of selections, just to qualify what I think my causal model implies,
“A or B, and if not those, then C, but not D or E”.
Actually, I think someone reading my forecast with weights will just leave with that kind of information anyway. If they try to mathematically apply the weights I chose to communicate my tiers of selections, then they will be led astray, expecting precision when there wasn’t any. They would do better to get details of the causal models involved and determine whether those have any merit, particularly in cases of:
very different forecasts (I forecast A or B, everyone else forecasts D)
a single forecast predicting very different outcomes (A or B are contradictory outcomes)
a homogenous bunch of forecasts (A, B, or A or B)
a heterogenous bunch of forecasts (A, B, C or D, A or B, E)
so basically in all cases. What might distinguish superforecasters is not their grasp of probability or their ability to update bayesian priors or whatever, but rather the applicability of causal models they develop, and what those causal models emphasize as causes and consequences.
That’s the background of my thinking, now here’s how I think it relates to what you’re saying:
If discoveries influence future outcomes in unknown ways, and your information is insufficient to predict all outcomes, then your causal model makes predictions that belong under an assumption of an open world. You are less useful as a predictor of outcomes and more useful as an supplier of possible outcomes. If we are both forecasting, and I supply outcomes A and B; you might supply outcomes C and D; someone else might supply E, F, and G; yet another person might supply H. Our forecasts run from A to H so far, and they are not exhaustive. As forecasters, our job becomes to create lists of plausible futures, not to select from predetermined lists.
I think this is appropriate to conditions where development of knowledge or inventions is a human choice. Any forecast will depend not only on what is plausible under some causal model, but also on what future people want to explore and how they explore it. Forecasts in that scenario can influence the future, so better that they supply options rather than weight them.
Well, my basic opinion about forecasting is that probabilities don’t inform the person receiving the forecast. Before you commit to weighting possible outcomes, you commit to at least two mutually exclusive futures, X and not X. So what you supply is a limitation on possible outcomes, either X or not X. At best, you’re aware of mutually exclusive alternative and specific futures. Then you can limit what not X means to something specific, for example, Y. So now you can say, “The future will contain X or Y.” That sort of analysis is enabled by your causal model. As your causal model improves, it becomes easier to supply a list of alternative future outcomes.
However, the future is not a game of chance, and there’s no useful interpretation to supply meaningful weights to the future prediction of any specific outcome, unless the outcomes belong to a game of chance, where you’re predicting rolls of a fair die, choice of a hand from a deck of cards, etc.
What’s worse, that does not limit your feelings about what probabilities apply. Those feelings can seem real and meaningful because they let you talk about lists of outcomes and which you think are more credible.
As a forecaster, I might supply outcomes in a forecast that I consider less credible along with those that I consider more credible. but if you ask me which options I consider credible, I might offer a subset of the list. So in that way weights can seem valuable, because they let you distinguish which you think are more credible and which you can rule out. But the weights also obscure that information because they can scale that credibility in confusing ways.
For example, I believe in outcomes A or B, but I offer A at 30%, B at 30%, C at 20%, D at 10%, and E at 10%. Have I communicated what I intended with my weights, namely, that A and B are credible, that C is somewhat credible, but D and E are not? Maybe I could adjust A and B to 40% and 40%, but now I’m fiddling with the likelihoods of C, D, and E, when all I really mean to communicate is that I like A or B as outcomes and C as an alternate. My probabilities communicate more and differently than I intend. I could make it clear with A and B each at 48% or something, but really now I’m trying to pretend I know what the chances of C, D, and E are, when all I really know about them is that my causal model doesn’t support their production much. I could go back and quantify that somehow, but information with which to do that is not available , so I have to pretend confidence in some estimation of the outcomes C, D, and E. My information is not useless, but it’s not relevant to weighting all possible outcomes against each other. If I’m forced to provide weights for all the listed outcomes, then I’m forced to figure out how to communicate my analysis in terms of weights so that the audience for my forecast understands what I intend to mean.
In general, analyzing causal models that determine possible futures is a distinct activity from weighting those futures. The valuable information is in the causal models and in the selection of futures based on those models. The extra information on epistemic confidence is not useful and pretends more information than a forecaster likely has. I would go as far as two tiers of selections, just to qualify what I think my causal model implies,
“A or B, and if not those, then C, but not D or E”.
Actually, I think someone reading my forecast with weights will just leave with that kind of information anyway. If they try to mathematically apply the weights I chose to communicate my tiers of selections, then they will be led astray, expecting precision when there wasn’t any. They would do better to get details of the causal models involved and determine whether those have any merit, particularly in cases of:
very different forecasts (I forecast A or B, everyone else forecasts D)
a single forecast predicting very different outcomes (A or B are contradictory outcomes)
a homogenous bunch of forecasts (A, B, or A or B)
a heterogenous bunch of forecasts (A, B, C or D, A or B, E)
so basically in all cases. What might distinguish superforecasters is not their grasp of probability or their ability to update bayesian priors or whatever, but rather the applicability of causal models they develop, and what those causal models emphasize as causes and consequences.
That’s the background of my thinking, now here’s how I think it relates to what you’re saying:
If discoveries influence future outcomes in unknown ways, and your information is insufficient to predict all outcomes, then your causal model makes predictions that belong under an assumption of an open world. You are less useful as a predictor of outcomes and more useful as an supplier of possible outcomes. If we are both forecasting, and I supply outcomes A and B; you might supply outcomes C and D; someone else might supply E, F, and G; yet another person might supply H. Our forecasts run from A to H so far, and they are not exhaustive. As forecasters, our job becomes to create lists of plausible futures, not to select from predetermined lists.
I think this is appropriate to conditions where development of knowledge or inventions is a human choice. Any forecast will depend not only on what is plausible under some causal model, but also on what future people want to explore and how they explore it. Forecasts in that scenario can influence the future, so better that they supply options rather than weight them.
I love it. Creating lists of plausible outcomes is very valuable, we can leave alone to idea of assigning probabilities.