I think the most lightweight-but-still-useful forecasting operationalization I’d be excited about is something like
12/24/120 months from now, will I still be very excited about this grant?
12/24/120 months from now, will I be extremely excited about this grant?
This gets at whether people think it’s a good idea ex post, and also (if people are well-calibrated) can quantify whether people are insufficiently or too risk/ambiguity-averse, in the classic sense of the term.
This seems helpful to assess fund managers’ calibration and improve their own thinking and decision-making. It’s less likely to be useful for communicating their views transparently to one another, or to the community, and it’s susceptible to post-hoc rationalization. I’d prefer an oracle external to the fund, like “12 months from now, will X have a ≥7/10 excitement about this grant on a 1-10 scale?”, where X is a person trusted by the fund managers who will likely know about the project anyway, such that the cost to resolve the forecast is small.
I plan to encourage the funds to experiment with something like this going forward.
I think the most lightweight-but-still-useful forecasting operationalization I’d be excited about is something like
This gets at whether people think it’s a good idea ex post, and also (if people are well-calibrated) can quantify whether people are insufficiently or too risk/ambiguity-averse, in the classic sense of the term.
This seems helpful to assess fund managers’ calibration and improve their own thinking and decision-making. It’s less likely to be useful for communicating their views transparently to one another, or to the community, and it’s susceptible to post-hoc rationalization. I’d prefer an oracle external to the fund, like “12 months from now, will X have a ≥7/10 excitement about this grant on a 1-10 scale?”, where X is a person trusted by the fund managers who will likely know about the project anyway, such that the cost to resolve the forecast is small.
I plan to encourage the funds to experiment with something like this going forward.
I agree that your proposed operationalization is better for the stated goals, assuming similar levels of overhead.