Reason #6: People rarely offer extraordinarily low probabilities.
Whenever I ask someone how likely they think something is, they pretty much never give a probability less than .1% unless that something is religious in nature. Given this, it seems like people systematically over estimate low probabilities because they fail to consider probabilities such as 10^-7 or 10^-53.
Some people would argue models predicting a probability of at least 0.1 % should have significant weight, for example, at least 10 %, which would imply an expected probability of at least 0.01 % (= 1*10^-3*0.1). However, I have significant concerns about this kind of reasoning. I worry the weights of the models are close to arbitrary. For instance, in Bob Fischer’s book about comparing welfare across species, there seems to be only 1 line about the weights. “We assigned 30 percent credence to the neurophysiological model, 10 percent to the equality model, and 60 percent to the simple additive model”. People usually give weights that are at least 0.1/”number of models”, which is at least 3.33 % (= 0.1/3) for 3 models, when it is quite hard to estimate the weights. However, giving weights which are not much smaller than the uniform weight of 1/”number of models” could easily lead to huge mistakes. As a silly example, if I asked random people with age 7 about whether the gravitational force between 2 objects is proportional to “distance”^-2 (correct answer), “distance”^-20, or “distance”^-200, I imagine I would get a significant fraction picking the exponents of −20 and −200. Assuming 60 % picked −2, 20 % picked −20, and 20 % picked −200, one may naively conclude the mean exponent of −45.2 (= 0.6*(-2) + 0.2*(-20) + 0.2*(-200)) is reasonable. Yet, there is lots of empirical evidence against this which the respondants are not aware of. The right conclusion would be that the respondants have practically no idea about the right exponent because they would not be able to adequately justify their picks.
Thanks! I really like that example you gave about asking seven year olds. This is definitely a major criticism of mine of Bob Fischer’s book. It seems like different experts could easily have come to very different conclusions because of what models they choose or, as you point out, how they weigh them.
Thanks. Relatedly, you may be interested in the comments from me and Wladimir on this post from Bentham’s Bulldog arguing for the possibility of intense agony in many species.
Hi James. Nice points.
Some people would argue models predicting a probability of at least 0.1 % should have significant weight, for example, at least 10 %, which would imply an expected probability of at least 0.01 % (= 1*10^-3*0.1). However, I have significant concerns about this kind of reasoning. I worry the weights of the models are close to arbitrary. For instance, in Bob Fischer’s book about comparing welfare across species, there seems to be only 1 line about the weights. “We assigned 30 percent credence to the neurophysiological model, 10 percent to the equality model, and 60 percent to the simple additive model”. People usually give weights that are at least 0.1/”number of models”, which is at least 3.33 % (= 0.1/3) for 3 models, when it is quite hard to estimate the weights. However, giving weights which are not much smaller than the uniform weight of 1/”number of models” could easily lead to huge mistakes. As a silly example, if I asked random people with age 7 about whether the gravitational force between 2 objects is proportional to “distance”^-2 (correct answer), “distance”^-20, or “distance”^-200, I imagine I would get a significant fraction picking the exponents of −20 and −200. Assuming 60 % picked −2, 20 % picked −20, and 20 % picked −200, one may naively conclude the mean exponent of −45.2 (= 0.6*(-2) + 0.2*(-20) + 0.2*(-200)) is reasonable. Yet, there is lots of empirical evidence against this which the respondants are not aware of. The right conclusion would be that the respondants have practically no idea about the right exponent because they would not be able to adequately justify their picks.
Thanks! I really like that example you gave about asking seven year olds. This is definitely a major criticism of mine of Bob Fischer’s book. It seems like different experts could easily have come to very different conclusions because of what models they choose or, as you point out, how they weigh them.
Thanks. Relatedly, you may be interested in the comments from me and Wladimir on this post from Bentham’s Bulldog arguing for the possibility of intense agony in many species.