I wonder though: how would this affect expected value calculations? Doesn’t this have far-reaching consequences?
One thing I have always wondered about is how to aggregate predicted values that differ by orders of magnitude. E.g. person A’s best guess is that the value of x will be 10, person B’s guess is that it will be 10,000. Saying that the expected value of x is ~5,000 seems to lose a lot of information. For simple monetary betting, this seems fine. For complicated decision-making, I’m less sure.
Let’s work this example through together! (but I will change the quantities to 10 and 20 for numerical stability reasons)
One thing we need to be careful with is not mixing the implied beliefs with the object level claims.
In this case, person A’s claim that the value is mA=10 is more accurately a claim that the beliefs of person A can be summed up as some distribution over the positive numbers, eg a log normal with parameters μA=logmA and σA . So the density distribution of beliefs of A is fA=1xσA√2πexp[−(lnx−μA)22σ2A] (and similar for person B, with mB=20 ). The scale parameters σA,σB intuitively represent the uncertainty of person A and person B.
Taking σA=σB=0.1, these densities look like:
Note that the mean of these distributions is slightly displaced upwards from the median expμ. Concretely, the mean is computed as exp[μ+σ22], and equals 10.05 and 20.10 for person A and person B respectively.
To aggregate the distributions, we can use the generalization of the geometric mean of odds referred to in footnote [1] of the post.
According to that, the aggregated distribution has a density f=√fA⋅√fB∫√fA⋅√fB.
The plot of the aggregated density looks like:
I actually notice that I am very surprised about this—I expected the aggregate distribution to be bimodal, but here it seems to have a single peak.
For this particular example, a numerical approximation of the expected value seems to equal around 14.21 - which exactly equals the geometric mean of the means.
I am not taking away any solid conclusions from this exercise—I notice I am still very confused about how the aggregated distribution looks like, and I encountered serious numerical stability issues when changing the parameters, which make me suspect a bug.
Maybe a Monte Carlo approach for estimating the expected value would solve the stability issues—I’ll see if I can get around to that at some point.
EDIT: Diego Chicharro has pointed out to me that the expected value can be easily computed analytically in Mathematica.
The resulting expected value of the aggregated distribution is exp[μAσ2B+μBσ2A+σ2Aσ2Bσ2A+σ2B].
In the case where σ2A=σ2B=σ2 we have then that the expected value is exp[μAσ2+μBσ2+σ2σ2σ2+σ2]=exp[μA+μB+σ22]=√exp[μA+σ2/2]√exp[μB+σ2/2], which is exactly the geometric mean of the expected values of the individual predictions.
Interesting! Seems intuitively right.
I wonder though: how would this affect expected value calculations? Doesn’t this have far-reaching consequences?
One thing I have always wondered about is how to aggregate predicted values that differ by orders of magnitude. E.g. person A’s best guess is that the value of x will be 10, person B’s guess is that it will be 10,000. Saying that the expected value of x is ~5,000 seems to lose a lot of information. For simple monetary betting, this seems fine. For complicated decision-making, I’m less sure.
Let’s work this example through together! (but I will change the quantities to 10 and 20 for numerical stability reasons)
One thing we need to be careful with is not mixing the implied beliefs with the object level claims.
In this case, person A’s claim that the value is mA= 10 is more accurately a claim that the beliefs of person A can be summed up as some distribution over the positive numbers, eg a log normal with parameters μA=logmA and σA . So the density distribution of beliefs of A is fA=1xσA√2πexp[−(lnx−μA)22σ2A] (and similar for person B, with mB=20 ). The scale parameters σA,σB intuitively represent the uncertainty of person A and person B.
Taking σA=σB=0.1, these densities look like:
Note that the mean of these distributions is slightly displaced upwards from the median expμ. Concretely, the mean is computed as exp[μ+σ22], and equals 10.05 and 20.10 for person A and person B respectively.
To aggregate the distributions, we can use the generalization of the geometric mean of odds referred to in footnote [1] of the post.
According to that, the aggregated distribution has a density f=√fA⋅√fB∫√fA⋅√fB.
The plot of the aggregated density looks like:
I actually notice that I am very surprised about this—I expected the aggregate distribution to be bimodal, but here it seems to have a single peak.
For this particular example, a numerical approximation of the expected value seems to equal around 14.21 - which exactly equals the geometric mean of the means.
I am not taking away any solid conclusions from this exercise—I notice I am still very confused about how the aggregated distribution looks like, and I encountered serious numerical stability issues when changing the parameters, which make me suspect a bug.
Maybe a Monte Carlo approach for estimating the expected value would solve the stability issues—I’ll see if I can get around to that at some point.
Meanwhile, here is my code for the results above.
EDIT: Diego Chicharro has pointed out to me that the expected value can be easily computed analytically in Mathematica.
The resulting expected value of the aggregated distribution is exp[μAσ2B+μBσ2A+σ2Aσ2Bσ2A+σ2B].
In the case where σ2A=σ2B=σ2 we have then that the expected value is exp[μAσ2+μBσ2+σ2σ2σ2+σ2]=exp[μA+μB+σ22]=√exp[μA+σ2/2]√exp[μB+σ2/2], which is exactly the geometric mean of the expected values of the individual predictions.