RSS

Model uncertainty

TagLast edit: May 29, 2022, 6:03 PM by Leo

Model uncertainty is uncertainty surrounding a model itself, including the model’s internal uncertainty estimates.

Uncertainty about models

A useful model is one that is simple enough to be analyzed easily, while nevertheless being similar enough to reality that this analysis can be used as a basis for predictions about the actual world. Unfortunately, it can be difficult to judge whether a given model is in fact similar enough. Furthermore, even if some predictions based on a model come true, this does not necessarily mean that next prediction based on the model will also come true.

A classic illustration of the importance of using appropriate models, as well as the difficulty of noticing when a model is inappropriate, is the 2007 financial crisis. In the years leading up to the crisis, many financial actors made investment decisions on the basis of models that assumed economic stability. Once this simplifying assumption ceased to hold, it became clear that their models had not sufficiently matched reality, and that the outcome of their decisions would be disastrous.

One strategy for dealing with uncertainty about the appropriateness of models is to construct and weight the predictions of multiple diverse models, rather than relying on a single one. However, in cases of radical uncertainty, not even this method may be enough. It may be that we think that there is a chance that none of the models that we have been able to generate is appropriate, and that we need to factor in what could happen if that were the case. Obviously, it is very hard to say something about such an uncertain case, but it may be possible to say some things. For instance, in their paper “Probing the Improbable,” Toby Ord, Rafaela Hillerbrand, and Anders Sandberg argue that in cases where our models about certain low probability, high-risk events—such as existential risks—are wrong, the chance of disaster may be substantially higher than if the models are right.

Uncertainty within models

When using a model to make estimates, we will often be uncertain about what values the model’s numerical parameters should have.

For example, if we decide to use 80,000 Hours’ three-factor framework for selecting cause areas, we may be unsure of what value to assign to a given cause area’s tractability, or if we are attempting to to estimate the value of a donation to a bednet charity, we may be unsure how many cases of malaria are prevented per bednet distributed.

It is important to make such uncertainty clear, both so that our views can be be more easily challenged and improved by others and so that we can derive more nuanced conclusions from the models we use.

By plugging in probability distributions or confidence windows, rather than individual estimates, for the values of the parameters in a given model, we can calculate an output for the model that also reflects uncertainty. However, it is important to be careful when performing such calculations, since small mathematical or conceptual errors can easily lead to incorrect or misleading results. A good tool for avoiding these sorts of errors is Guesstimate.[1]

It has also been argued, e.g. by Holden Karnofsky, that in cases with high uncertainty, estimates that assign an intervention a very high expected value are likely to reflect some unseen bias in the calculation, and should therefore be treated with skepticism.

Further reading

Frigg, Roman & Stephan Hartmann (2006) Models in science, Stanford Encyclopedia of Philosophy, February 27 (updated 4 February 2020).

Karnofsky, Holden (2011) Why we can’t take expected value estimates literally (even when they’re unbiased), The GiveWell Blog, August 18 (updated 25 July 2016).
Approach to evaluating uncertain interventions.

Karnofsky, Holden (2014) Sequence thinking vs. cluster thinking, The GiveWell Blog, June 10 (updated 25 July 2016).

Ord, Toby, Rafaela Hillerbrand & Anders Sandberg (2010) Probing the improbable: methodological challenges for risks with low probabilities and high stakes, Journal of risk research, vol. 13, pp. 191–205.

External links

Guesstimate. A tool for carrying out calculations under uncertainty.

Related entries

credence | forecasting | sequence vs. cluster thinking | value of information

  1. ^

    Gooen, Ozzie (2015) Guesstimate: An app for making decisions with confidence (intervals), Effective Altruism Forum, December 30.

Se­quence think­ing vs. cluster thinking

GiveWellJul 25, 2016, 10:43 AM
17 points
0 comments28 min readEA link
(blog.givewell.org)

Hedg­ing against deep and moral uncertainty

MichaelStJulesSep 12, 2020, 11:44 PM
83 points
13 comments9 min readEA link

A prac­ti­cal guide to long-term plan­ning – and sug­ges­tions for longtermism

weeatquinceOct 10, 2021, 3:37 PM
140 points
13 comments24 min readEA link

Meth­ods for im­prov­ing un­cer­tainty anal­y­sis in EA cost-effec­tive­ness models

FroolowAug 29, 2022, 6:07 PM
127 points
8 comments37 min readEA link

“The Race to the End of Hu­man­ity” – Struc­tural Uncer­tainty Anal­y­sis in AI Risk Models

FroolowMay 19, 2023, 12:03 PM
48 points
4 comments21 min readEA link

Prior X%—<1%: A quan­tified ‘epistemic sta­tus’ of your pre­dic­tion.

tcelferactJun 2, 2023, 3:51 PM
11 points
1 comment1 min readEA link

X-risk Agnosticism

Richard Y Chappell🔸Jun 8, 2023, 3:02 PM
34 points
1 comment5 min readEA link
(rychappell.substack.com)

How We Plan to Ap­proach Uncer­tainty in Our Cost-Effec­tive­ness Models

GiveWellJan 2, 2024, 9:09 PM
92 points
26 comments23 min readEA link
(www.givewell.org)

Even Allo­ca­tion Strat­egy un­der High Model Ambiguity

MichaelStJulesDec 31, 2020, 9:10 AM
15 points
3 comments2 min readEA link

Seek­ing feed­back on new EA-al­igned eco­nomics paper

jhOct 21, 2021, 9:19 PM
35 points
13 comments1 min readEA link

‘Dis­solv­ing’ AI Risk – Pa­ram­e­ter Uncer­tainty in AI Fu­ture Forecasting

FroolowOct 18, 2022, 10:54 PM
111 points
63 comments39 min readEA link

EAs un­der­es­ti­mate un­cer­tainty in cause prioritisation

freedomandutilityAug 23, 2022, 2:04 PM
119 points
20 comments2 min readEA link

How do EA Orgs Ac­count for Uncer­tainty in their Anal­y­sis?

Peter WildefordApr 5, 2017, 4:48 PM
17 points
3 comments15 min readEA link

How to eval­u­ate rel­a­tive im­pact in high-un­cer­tainty con­texts? An up­date on re­search method­ol­ogy & grant­mak­ing of FP Cli­mate

jackvaMay 26, 2023, 5:30 PM
84 points
8 comments16 min readEA link

Prob­a­bil­ity dis­tri­bu­tions of Cost-Effec­tive­ness can be misleading

Lorenzo Buonanno🔸Jul 18, 2022, 5:42 PM
70 points
49 comments1 min readEA link

My notes on: Se­quence think­ing vs. cluster thinking

Vasco Grilo🔸May 25, 2022, 3:03 PM
24 points
0 comments5 min readEA link

Why GiveWell should use com­plete un­cer­tainty quantification

TanaeDec 27, 2022, 8:11 PM
32 points
1 comment1 min readEA link
(suboptimal.substack.com)

[Question] Re­quest for As­sis­tance—Re­search on Sce­nario Devel­op­ment for Ad­vanced AI Risk

KiliankMar 30, 2022, 3:01 AM
2 points
1 comment1 min readEA link

Re­fut­ing longter­mism with Fer­mat’s Last Theorem

astuppleAug 16, 2022, 12:26 PM
3 points
32 comments3 min readEA link

Quan­tify­ing Uncer­tainty in GiveWell’s GiveDirectly Cost-Effec­tive­ness Analysis

SamNolanMay 27, 2022, 3:10 AM
130 points
19 comments6 min readEA link

A model about the effect of to­tal ex­is­ten­tial risk on ca­reer choice

Jonas MossSep 10, 2022, 7:18 AM
12 points
4 comments2 min readEA link

Why does GiveWell not provide lower and up­per es­ti­mates for the cost-effec­tive­ness of its top char­i­ties?

Vasco Grilo🔸Jul 31, 2022, 1:41 PM
45 points
8 comments1 min readEA link

Be less trust­ing of in­tu­itive ar­gu­ments about so­cial phe­nom­ena

Nathan_BarnardDec 18, 2022, 1:11 AM
43 points
20 comments4 min readEA link

Model-Based Policy Anal­y­sis un­der Deep Uncertainty

UtilonMar 6, 2023, 2:24 PM
103 points
31 comments21 min readEA link

Re­boot­ing AI Gover­nance: An AI-Driven Ap­proach to AI Governance

UtilonMay 20, 2023, 7:06 PM
38 points
4 comments30 min readEA link

Assess­ment of AI safety agen­das: think about the down­side risk

Roman LeventovDec 19, 2023, 9:02 AM
6 points
0 comments1 min readEA link
No comments.