Agreed that there are some contexts where there’s more value in getting distributions, like with the Fermi paradox.
Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
> Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
Aha, but you can also do this when the final answer is also a distribution. In particular, you can look at the KL-divergence between the initial distribution and the answer, and this is also a proper scoring rule.
Agreed that there are some contexts where there’s more value in getting distributions, like with the Fermi paradox.
Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
> Or, before the grants are given out, you could ask people to give an ex ante distribution for “what will be your ex post point estimate of the value of this grant?” That feeds directly into VOI calculations, and it is clearly defined what the distribution represents. But note that it requires focusing on point estimates ex post.
Aha, but you can also do this when the final answer is also a distribution. In particular, you can look at the KL-divergence between the initial distribution and the answer, and this is also a proper scoring rule.