I donāt think Anders Sandberg uses the EA Forum, so Iāll just repost what Anders wrote in reaction to this on Twitter:
āI suspect we have a āpublication biasā of tech predictions where the pessimists donāt make predictions (think the tech impossible or irrelevant, hence donāt respond to queries, or find their long timescales so uncertain they are loath to state them).
In this case it is fairly clear that progress is being made but it is slower than hoped for: predictions as a whole made a rate mistake, but perhaps not an eventual outcome mistake (we will see). I think this is is[sic] a case of Amaraās law.
Amaraās law (that we overestimate the magnitude of short-term change and underestimate long-term change) can be explained by exponential-blindness, but also hype cycles, and integrating a technology in society is a slow processā
Fwiw, I broadly agree. I think those in the industry making public predictions have plausibly āgoodā reasons to skew optimistic. Attracting the funding, media attention, talent necessary to make progress might simply require generating buzz and optimism- even if the progress it generates is at a slower rate that implied by their public predictions. So it would actually be odd if overall the majority of predictions by these actors donāt resolve negatively and overly optimistic (they arenāt trying to rank high on the Metaculus leaderboard).
So those who are shocked by the results presented here may have cause to update and put less weight on predictions from cultured media companies and the media repeating them, and rely on something else. For those who arenāt surprised by these results, then they probably already placed an appropriate weight on how seriously to take public predictions from the industry.
On how this industryās predictions compare to othersā, I too would like to see that and identify the right reference class(es).
I think it should be pretty clear that there are a ton of biases going on. In Expert Political Judgement, there was a much earlier study on expert/āpundit forecasting ability, and the results were very poor. I donāt see reasons why we should have expected different here.
One thing that might help would be āmeta-forecastingā. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. Iād predict that they would have given pretty poor scores to most of these groups, especially ācompanies making public claims about their own technologiesā, and āmagazines and public mediaā (which also seem just as biased).
One thing that might help would be āmeta-forecastingā. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. Iād predict that they would have given pretty poor scores to most of these groups.
I donāt think Anders Sandberg uses the EA Forum, so Iāll just repost what Anders wrote in reaction to this on Twitter:
āI suspect we have a āpublication biasā of tech predictions where the pessimists donāt make predictions (think the tech impossible or irrelevant, hence donāt respond to queries, or find their long timescales so uncertain they are loath to state them).
In this case it is fairly clear that progress is being made but it is slower than hoped for: predictions as a whole made a rate mistake, but perhaps not an eventual outcome mistake (we will see). I think this is is[sic] a case of Amaraās law.
Amaraās law (that we overestimate the magnitude of short-term change and underestimate long-term change) can be explained by exponential-blindness, but also hype cycles, and integrating a technology in society is a slow processā
Fwiw, I broadly agree. I think those in the industry making public predictions have plausibly āgoodā reasons to skew optimistic. Attracting the funding, media attention, talent necessary to make progress might simply require generating buzz and optimism- even if the progress it generates is at a slower rate that implied by their public predictions. So it would actually be odd if overall the majority of predictions by these actors donāt resolve negatively and overly optimistic (they arenāt trying to rank high on the Metaculus leaderboard).
So those who are shocked by the results presented here may have cause to update and put less weight on predictions from cultured media companies and the media repeating them, and rely on something else. For those who arenāt surprised by these results, then they probably already placed an appropriate weight on how seriously to take public predictions from the industry.
On how this industryās predictions compare to othersā, I too would like to see that and identify the right reference class(es).
I think it should be pretty clear that there are a ton of biases going on. In Expert Political Judgement, there was a much earlier study on expert/āpundit forecasting ability, and the results were very poor. I donāt see reasons why we should have expected different here.
One thing that might help would be āmeta-forecastingā. We could later have some expert forecasters predict the accuracy of average statements made by different groups in different domains. Iād predict that they would have given pretty poor scores to most of these groups, especially ācompanies making public claims about their own technologiesā, and āmagazines and public mediaā (which also seem just as biased).
I agree with your meta-meta-forecast.