In terms of forecasting accuracy on Metaculus, Eli’s individual performance is comparable[1] to the community aggregate on his own, despite him having optimised for volume (he’s 10th on the heavily volume weighted leaderboard). I expect that were he to have pushed less hard for volume, he’d have significantly outperformed the community aggregate even as an individual.[2]
Assuming the other Samotsvety forecasters are comparably good, I’d expect the aggregated forecasts from the group to very comfortably outperform the community aggregate, even if they weren’t paying unusual attention to the questions (which they are).
Comparing ‘score at resolution time’, Eli looks slightly worse than the community. Comparing ‘score across all times’, Eli looks better than the community. Score across all times is a better measure of skill when comparing individuals, but does disadvantage the community prediction, because at earlier times questions have fewer predictors.
As some independent evidence of this, I comfortably outperform the community aggregate, having tried less hard than Eli to optimise for volume. Eli has beaten me in more than one competition, and think he’s a better forecaster.
In terms of forecasting accuracy on Metaculus, Eli’s individual performance is comparable[1] to the community aggregate on his own, despite him having optimised for volume (he’s 10th on the heavily volume weighted leaderboard). I expect that were he to have pushed less hard for volume, he’d have significantly outperformed the community aggregate even as an individual.[2]
Assuming the other Samotsvety forecasters are comparably good, I’d expect the aggregated forecasts from the group to very comfortably outperform the community aggregate, even if they weren’t paying unusual attention to the questions (which they are).
Comparing ‘score at resolution time’, Eli looks slightly worse than the community. Comparing ‘score across all times’, Eli looks better than the community. Score across all times is a better measure of skill when comparing individuals, but does disadvantage the community prediction, because at earlier times questions have fewer predictors.
As some independent evidence of this, I comfortably outperform the community aggregate, having tried less hard than Eli to optimise for volume. Eli has beaten me in more than one competition, and think he’s a better forecaster.