> Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
Aha, but you can also do this when the final answer is also a distribution. In particular, you can look at the KL-divergence between the initial distribution and the answer, and this is also a proper scoring rule.
> Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
Aha, but you can also do this when the final answer is also a distribution. In particular, you can look at the KL-divergence between the initial distribution and the answer, and this is also a proper scoring rule.