Theoretical idea that could be implemented into Metaculus
tldr; add an option to submit models of how to forecast a question, and also voting on the models.
To be more concrete, when someone submits a question, in addition to forecasting the question, you can submit a squiggle—or just plain mathematical model—of your best current guess of how to approach the problem. You define each subcomponent that is important in the final forecast and also how these subcomponents combine into the final forecast. Each subcomponent automatically becomes another forecasting question on the site that people can do the same to (if it is not already one).
Then in addition to a normal forecast, as we do right now, people can also forecast the subcomponents of the models, as well as vote on the models. If a model already includes previously forecasted questions, they automatically populate in the model.
The voting system on models could either just draw attention to the best models and encourage forecasting of the subcomponents, or even weight the models estimates into the overall forecast of the question. No idea if this would improve forecasting but it might make it more transparent and scalable.
I wrote a bit more in this google doc if interested.
edit: I think this might just be guesstimate with memoization
Theoretical idea that could be implemented into Metaculus
tldr; add an option to submit models of how to forecast a question, and also voting on the models.
To be more concrete, when someone submits a question, in addition to forecasting the question, you can submit a squiggle—or just plain mathematical model—of your best current guess of how to approach the problem. You define each subcomponent that is important in the final forecast and also how these subcomponents combine into the final forecast. Each subcomponent automatically becomes another forecasting question on the site that people can do the same to (if it is not already one).
Then in addition to a normal forecast, as we do right now, people can also forecast the subcomponents of the models, as well as vote on the models. If a model already includes previously forecasted questions, they automatically populate in the model.
The voting system on models could either just draw attention to the best models and encourage forecasting of the subcomponents, or even weight the models estimates into the overall forecast of the question. No idea if this would improve forecasting but it might make it more transparent and scalable.
I wrote a bit more in this google doc if interested.
edit: I think this might just be guesstimate with memoization