This is a relevant question if you’re thinking about how hard you should try to drive engagement on a forecasting question.
What is the ‘policy relevance’ of answering the title question? Ie. if the answer is “yes, forecaster count strongly increases accuracy”, how would you go about increasing the number of forecasters?
For Metaculus there are lots of ways to drive engagement: prioritise making the platform easier to use, increase cash prizes, community building and outreach etc.
But as mentioned in the article the problem in practice is that the bootstrap answer is probably misleading, as increasing the number of forecasters likely changes forecaster composition.
However, one specific example where the analysis might be actually applicable is when you’re thinking about how many Pro Forecasters you hire for a job.
What is the ‘policy relevance’ of answering the title question? Ie. if the answer is “yes, forecaster count strongly increases accuracy”, how would you go about increasing the number of forecasters?
For Metaculus there are lots of ways to drive engagement: prioritise making the platform easier to use, increase cash prizes, community building and outreach etc.
But as mentioned in the article the problem in practice is that the bootstrap answer is probably misleading, as increasing the number of forecasters likely changes forecaster composition.
However, one specific example where the analysis might be actually applicable is when you’re thinking about how many Pro Forecasters you hire for a job.