I think it should be pretty clear that there are a ton of biases going on. In Expert Political Judgement, there was a much earlier study on expert/pundit forecasting ability, and the results were very poor. I don’t see reasons why we should have expected different here.
One thing that might help would be “meta-forecasting”. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. I’d predict that they would have given pretty poor scores to most of these groups, especially “companies making public claims about their own technologies”, and “magazines and public media” (which also seem just as biased).
One thing that might help would be “meta-forecasting”. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. I’d predict that they would have given pretty poor scores to most of these groups.
I think it should be pretty clear that there are a ton of biases going on. In Expert Political Judgement, there was a much earlier study on expert/pundit forecasting ability, and the results were very poor. I don’t see reasons why we should have expected different here.
One thing that might help would be “meta-forecasting”. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. I’d predict that they would have given pretty poor scores to most of these groups, especially “companies making public claims about their own technologies”, and “magazines and public media” (which also seem just as biased).
I agree with your meta-meta-forecast.