Motte: We should give $225k to Metaculus every year
Bailey: This very specific method of estimating the value of Metaculus questions leads to a back-of-the-napkin guess that Metaculus questions might be worth on the order of $225k/year to the EA community, but I could imagine this being an overestimate, particularly if nobody ends up changing any decisions because of Metaculus predictions, or if the estimate of $2000 per highly valuable question is too high.
What I actually think: I think that Metaculus is great, but I was worried that their questions might not have any effects on decisions, and thus ultimately not be valuable. After a brief investigation, I think that a fair number of its questions are valuable. To incentivize questions that the EA community finds valuable, and to ensure that Metaculus remains on a good financial footing, the EA community could each year try to estimate the value Metaculus questions produce and pay Metaculus (proportionally to) that amount. I think this would require a bit more effort than my back-of-the-napkin calculation right there, but not that much if EA is still vetting constrained.
I wonder how valuable the training and talent selection effects are for the community. RP seems to think they are valuable, e.g. they targeted Metaculus forecasters specifically for their current hiring round. Maybe RP people can estimate how valuable legible forecasting experience and skill is to them?
Also, forecasting is a concrete and useful practice related to EA that anyone can do and skill up in. E.g. the EA forecasting tournament last year was a great shared experience for my local chapter (not only cause we won :P).
Motte: We should give $225k to Metaculus every year
Bailey: This very specific method of estimating the value of Metaculus questions leads to a back-of-the-napkin guess that Metaculus questions might be worth on the order of $225k/year to the EA community, but I could imagine this being an overestimate, particularly if nobody ends up changing any decisions because of Metaculus predictions, or if the estimate of $2000 per highly valuable question is too high.
What I actually think: I think that Metaculus is great, but I was worried that their questions might not have any effects on decisions, and thus ultimately not be valuable. After a brief investigation, I think that a fair number of its questions are valuable. To incentivize questions that the EA community finds valuable, and to ensure that Metaculus remains on a good financial footing, the EA community could each year try to estimate the value Metaculus questions produce and pay Metaculus (proportionally to) that amount. I think this would require a bit more effort than my back-of-the-napkin calculation right there, but not that much if EA is still vetting constrained.
I wonder how valuable the training and talent selection effects are for the community. RP seems to think they are valuable, e.g. they targeted Metaculus forecasters specifically for their current hiring round. Maybe RP people can estimate how valuable legible forecasting experience and skill is to them?
Also, forecasting is a concrete and useful practice related to EA that anyone can do and skill up in. E.g. the EA forecasting tournament last year was a great shared experience for my local chapter (not only cause we won :P).
Good question; I am not sure what the answer would be.