I like the approach to Pascalās Mugging described in this post from GiveWell (search for āPascalās Mugging refersā). Based on this analysis from Dario Amodei, given a prior X1 and estimate X2, the expected posterior can be calculated in a bayesian way via the inverse-variance approach:
E(X)=E(X1)V(X1)+E(X2)V(X2)1V(X1)+1V(X2), where E and V stand for the expect value and variance.
The above implies that, if the estimate X2 is much more uncertain than the prior X1 (i.e. V(X2)>>V(X1)), E(X) is roughly equal to E(X1). As a result, if someone asks me on the street for 1 $, promising 1 T$ (= E(X2)) in return, it makes sense to ignore the promise. the uncertainty of my prior (e.g. E(X1)= ā1 $, and V(X1) = 1 $) would be much smaller than the uncertainty of the promise (V(X2) >> 1 $).
I think your āreversal/āinconsistency testā is a way of assessing V(X2). āIf the logic seems to lead to taking action X [i.e. being Pascal-mugged], and seems to equally validly lead to taking an action inconsistent with Xā, the variance of the estimate is probably much larger than that of the prior (V(X2)>>V(X1)).
Hi Sanjay,
I like the approach to Pascalās Mugging described in this post from GiveWell (search for āPascalās Mugging refersā). Based on this analysis from Dario Amodei, given a prior X1 and estimate X2, the expected posterior can be calculated in a bayesian way via the inverse-variance approach:
E(X)=E(X1)V(X1)+E(X2)V(X2)1V(X1)+1V(X2), where E and V stand for the expect value and variance.
The above implies that, if the estimate X2 is much more uncertain than the prior X1 (i.e. V(X2)>>V(X1)), E(X) is roughly equal to E(X1). As a result, if someone asks me on the street for 1 $, promising 1 T$ (= E(X2)) in return, it makes sense to ignore the promise. the uncertainty of my prior (e.g. E(X1)= ā1 $, and V(X1) = 1 $) would be much smaller than the uncertainty of the promise (V(X2) >> 1 $).
I think your āreversal/āinconsistency testā is a way of assessing V(X2). āIf the logic seems to lead to taking action X [i.e. being Pascal-mugged], and seems to equally validly lead to taking an action inconsistent with Xā, the variance of the estimate is probably much larger than that of the prior (V(X2)>>V(X1)).