Thanks for sharing this, awareness of this type of bias is very relevant for the EA community.
The interpretation of $\sigma_V / \sigma_\mu$ (squared) is subtle in practice. I think a clean way to express it is the (square root of the) ratio of prior precision to “measurement” precision—that fits with the hierarchical model used to explain it in the paper you reference.
In practice this is not trivial to guesstimate.
An interesting rabbit hole to understand this further is the “Tweedie correction” [1].
It should also be pointed out that once you’ve shrunk the estimate, that’s it: EV maximising will pick the posterior winner without accounting for the posterior variance—also something not everyone is comfortable with.
Thanks for sharing this, awareness of this type of bias is very relevant for the EA community.
The interpretation of $\sigma_V / \sigma_\mu$ (squared) is subtle in practice. I think a clean way to express it is the (square root of the) ratio of prior precision to “measurement” precision—that fits with the hierarchical model used to explain it in the paper you reference.
In practice this is not trivial to guesstimate.
An interesting rabbit hole to understand this further is the “Tweedie correction” [1].
It should also be pointed out that once you’ve shrunk the estimate, that’s it: EV maximising will pick the posterior winner without accounting for the posterior variance—also something not everyone is comfortable with.
[1] https://efron.ckirby.su.domains/papers/2011TweediesFormula.pdf