On the one hand, my opinion of Metaculus predictions worsened as I saw how the ‘recent predictions’ showed people piling in on the median on some questions I watch.
Can you say more about this? I ask because this behavior seems consistent with an attitude of epistemic deference towards the community prediction when individual predictors perceive it to be superior to what they can themselves predict given their time and ability constraints.
Sure at an individual level deference usually makes for better predictions, but at a community level deference-as-the-norm can dilute the weight of those who are informed and predict differently from the median. Excessive numbers of deferential predictions also obfuscate how reliable the median prediction is, and thus makes it harder for others to do an informed update on the median.
As you say, it’s better if people contribute information where their relative value-add is greatest, so I’d say it’s reasonable for people to have a 2:1 ratio of questions on which they deviate from the median to questions on which they follow the median. My vague impression is that the ratio may be lower—especially for people predicting on <1 year time horizon events. I think you, linch and other heavier Metaculus users may have a more informed impression here though, so would be happy to see disagreement.
I think it would be interesting to have a Metaculus on which for every prediction you have to select a general category for your update e.g. “New Probability Calculation”, “Updated to Median”, “Information source released”, etc. Seeing the various distributions for each would likely be quite informative.
Can you say more about this? I ask because this behavior seems consistent with an attitude of epistemic deference towards the community prediction when individual predictors perceive it to be superior to what they can themselves predict given their time and ability constraints.
Sure at an individual level deference usually makes for better predictions, but at a community level deference-as-the-norm can dilute the weight of those who are informed and predict differently from the median. Excessive numbers of deferential predictions also obfuscate how reliable the median prediction is, and thus makes it harder for others to do an informed update on the median.
As you say, it’s better if people contribute information where their relative value-add is greatest, so I’d say it’s reasonable for people to have a 2:1 ratio of questions on which they deviate from the median to questions on which they follow the median. My vague impression is that the ratio may be lower—especially for people predicting on <1 year time horizon events. I think you, linch and other heavier Metaculus users may have a more informed impression here though, so would be happy to see disagreement.
I think it would be interesting to have a Metaculus on which for every prediction you have to select a general category for your update e.g. “New Probability Calculation”, “Updated to Median”, “Information source released”, etc. Seeing the various distributions for each would likely be quite informative.