I think this misunderstands forecasting or mischaracterises decision making.
Your brain absolutely does decide when to make a left-hand-turn at a busy intersection based on a probabilistic estimate. Your brain is just pretty reliable at making said judgments that you assume some binary choice was made. Subconciously you are weighing up the likelihood of various risks based on your senses observing them and concluding that there is a very low (or an appropriately low) probability of it going wrong. The easy way to test this is go to a zip line, one with a drunk operator and one with a sober operator. The zip line may look identically safe but your brain will (hopefully) consider the former more risky—not because the impact of something going wrong is different between the zip lines (both falls would hurt, if not kill you), but because the likelihood of something going wrong is different (as you’d calculate that a drunk operator may not be as cognitively aware of what they are doing). That’s a probabilistic assessment.
The difference is, most decisions we deal with in the world don’t need us to sit down and do a formal prediction process to ensure good decision making. However, most organisational, and especially those on policy and governance etc., do require it. They are concretely two predictions you are making:
1. What will the world be? (through the lens of what you care about, e.g. will there be an oil crisis?)
2. What influence on the world will my policy have? (e.g. if I send Trump a friendly email will he avoid attacking another petrostate?)
Viewing forecasting as just the probabilistic estimate is missing the entire benefit. Forecasting is the entire process. Stating your view probabilistic just allows you to understand your own and others uncertainty (and to provide a purity of accountability and incentive alignment). It only adds decision fatigue if you have not put in the right processes to interpret the result (such as upfront thresholds for action). If you are following an optimal predictive decision making process, you should be making your assumptions and your weighing up of information explicit. This is how you determine what is relevant, what isn’t relevant etc.
Dominic Cummings statement misses that everything he said there IS forecasting. What he’s actually saying is “the point estimate from forecasters was not useful—it was the explicit reasoning about the causal chain etc. from forecasters that was useful”.
However, benchmarking against accuracy ensures the incentives are correct and that decisions aren’t manipulated by elements that decrease the end outcomes efficacy.
If you conclude that it’d be better if decision makers had: 1) the most accurate view of the world; and 2) the most accurate view of how their actions may influence that world towards their objectives. Then I would stand by the statement that it’s sad.
I think this misunderstands forecasting or mischaracterises decision making.
Your brain absolutely does decide when to make a left-hand-turn at a busy intersection based on a probabilistic estimate. Your brain is just pretty reliable at making said judgments that you assume some binary choice was made. Subconciously you are weighing up the likelihood of various risks based on your senses observing them and concluding that there is a very low (or an appropriately low) probability of it going wrong. The easy way to test this is go to a zip line, one with a drunk operator and one with a sober operator. The zip line may look identically safe but your brain will (hopefully) consider the former more risky—not because the impact of something going wrong is different between the zip lines (both falls would hurt, if not kill you), but because the likelihood of something going wrong is different (as you’d calculate that a drunk operator may not be as cognitively aware of what they are doing). That’s a probabilistic assessment.
The difference is, most decisions we deal with in the world don’t need us to sit down and do a formal prediction process to ensure good decision making. However, most organisational, and especially those on policy and governance etc., do require it. They are concretely two predictions you are making:
1. What will the world be? (through the lens of what you care about, e.g. will there be an oil crisis?)
2. What influence on the world will my policy have? (e.g. if I send Trump a friendly email will he avoid attacking another petrostate?)
Viewing forecasting as just the probabilistic estimate is missing the entire benefit. Forecasting is the entire process. Stating your view probabilistic just allows you to understand your own and others uncertainty (and to provide a purity of accountability and incentive alignment). It only adds decision fatigue if you have not put in the right processes to interpret the result (such as upfront thresholds for action). If you are following an optimal predictive decision making process, you should be making your assumptions and your weighing up of information explicit. This is how you determine what is relevant, what isn’t relevant etc.
Dominic Cummings statement misses that everything he said there IS forecasting. What he’s actually saying is “the point estimate from forecasters was not useful—it was the explicit reasoning about the causal chain etc. from forecasters that was useful”.
However, benchmarking against accuracy ensures the incentives are correct and that decisions aren’t manipulated by elements that decrease the end outcomes efficacy.
If you conclude that it’d be better if decision makers had: 1) the most accurate view of the world; and 2) the most accurate view of how their actions may influence that world towards their objectives. Then I would stand by the statement that it’s sad.