Update: the Forecasting Research Institute has changed the language in the report in response to this critique! (It seems like titotal played an important role in this. Thank you, titotal.)
On page 32, the report now gives the survey results in the same intersubjective resolution/metaprediction wording the question was asked in, rather than as an unqualified probability:
By 2030, the average expert thinks that 23% of LEAP panelists will say the state of AI most closely mirrors an (“rapid”) AI progress scenario that matches some of these claims.
This is awesome! I’m very happy to see this. Thanks to the Forecasting Research Institute for making this change.
I see also this EA Forum post has been updated in the same way, so thanks for that as well. Great to see.
Update #2: titotal has published a full breakdown of the error involving the intersubjective resolution/metaprediction framing of the survey question. It’s a great post that explains the error very well. Many thanks to titotal for taking the time to write the post and for talking to the Forecasting Research Institute about this. Thanks again to the Forecasting Research Institute for revising the report and this post.
Update: the Forecasting Research Institute has changed the language in the report in response to this critique! (It seems like titotal played an important role in this. Thank you, titotal.)
On page 32, the report now gives the survey results in the same intersubjective resolution/metaprediction wording the question was asked in, rather than as an unqualified probability:
This is awesome! I’m very happy to see this. Thanks to the Forecasting Research Institute for making this change.
I see also this EA Forum post has been updated in the same way, so thanks for that as well. Great to see.
Update #2: titotal has published a full breakdown of the error involving the intersubjective resolution/metaprediction framing of the survey question. It’s a great post that explains the error very well. Many thanks to titotal for taking the time to write the post and for talking to the Forecasting Research Institute about this. Thanks again to the Forecasting Research Institute for revising the report and this post.