Last I checked, Tetlockâs result on the efficacy of superforecasters vs. domain experts wasnât apples-to-apples: it was comparing individual domain expert forecasts vs. superforecaster forecasts that had been aggregated.
As this post explains, the main study that people cite when saying that âsuperforecasters are better than expertsâ comes from a competition where the aggregation methods for the two groups was different (Good Judgment Projectâs aggregation algorithm versus prediction market with low liquidity for amateur forecasters and experts, respectively). Prediction markets for forecasters and experts had similar performance.
Last I checked, Tetlockâs result on the efficacy of superforecasters vs. domain experts wasnât apples-to-apples: it was comparing individual domain expert forecasts vs. superforecaster forecasts that had been aggregated.
As this post explains, the main study that people cite when saying that âsuperforecasters are better than expertsâ comes from a competition where the aggregation methods for the two groups was different (Good Judgment Projectâs aggregation algorithm versus prediction market with low liquidity for amateur forecasters and experts, respectively). Prediction markets for forecasters and experts had similar performance.