Ten Commandments for Aspiring Superforecasters

[Vasco Grilo]

Link post

This is a linkpost for Ten Commandments for Aspiring Superforecasters by Good Judgment^[1]. I think the lessons also apply outside of explicit forecasting.

In Superforecasting: The Art and Science of Prediction, Good Judgment co-founder Philip Tetlock and his co-author Dan Gardner summarize the Good Judgment Project research findings in the form of “Ten Commandments for Aspiring Superforecasters.” These commandments describe behaviors that have been “experimentally demonstrated to boost [forecasting] accuracy.” You can learn more about these commandments—and practice applying them under the guidance of professional Superforecasters—at one of Good Judgment’s training workshops.

Here are the 10 commandments:

1. Triage. “Focus on questions where your hard work is likely to pay off”.

2. Break seemingly intractable problems into tractable sub-problems.

3. Strike the right balance between inside and outside views.

4. Strike the right balance between under- and overreacting to evidence.

5. Look for the clashing causal forces at work in each problem.

6. Strive to distinguish as many degrees of doubt as the problem permits but no more.

7. Strike the right balance between under- and overconfidence, between prudence and decisiveness.

8. Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases.

9. Bring out the best in others and let others bring out the best in you.

10. Master the error-balancing bicycle. “Implementing each commandment requires balancing opposing errors”.

Last but not least, there is the “11th commandment”:

“It is impossible to lay down binding rules,” Helmuth von Moltke warned, “because two cases will never be exactly the same.”

1. ^

   I wanted this to be a crosspost, but it was not possible due to copyright restrictions.

[MathiasKB]

I think this leaves out what is perhaps the most important step in making a quality forecast, which is to consider the baserates!

[Vasco Grilo]

Nice point, Mathias! I agree reference class forecasting is super important. I think it is supposed to be included in the 3rd commandment about the inside and outside view:

Superforecasters know that there is nothing new under the sun. Nothing is 100% “unique.” Language purists be damned: uniqueness is a matter of degree. So Superforecasters conduct creative searches for comparison classes even for seemingly unique events, such as the outcome of a hunt for a high-profile terrorist (Joseph Kony) or the standoff between a new socialist government in Athens and Greece’s creditors. Superforecasters are in the habit of posing the outside-view question: How often do things of this sort happen in situations of this sort?