Surveying scientific experts seems like a promising way to gather informed opinions relevant for important decisions, and to improve common knowledge on important topics.
The IGM economic experts panel has been doing surveys for roughly 10 years, and I wonder how well both the surveyed consensus and the format of the platform hold up, as it might be a good idea to
copy the general IGM platform to other scientific fields,
offer ideas to improve the IGM panel, and more ambitiously to improve the economics consensus (e.g. by spotting biases), and
use the survey results when they interact with topics relevant for EA.
Example for recent survey responses
Two ideas how to evaluate the survey’s usefulness
Look through IGM’s survey questions and evaluate those which have something like a “correct” answer in retrospect. This requires somebody fairly informed about the surveyed issues.
Interview someone from IGM, or a researcher knowledgeable about expert consensuses in economics or generally (suggestions?).
Two articles about the IGM survey
Tyler Cowen evaluating some IGM polls in 2015: “But quite seriously, my opinion of the professional consensus — on topics outside an individual’s research specialty — really has gone down as a result of these polls.” (link)
discussion of biases in the economics elite, in the surveyed IGM questions and for which experts were selected (paper from 2016)
IGM asking softball questions apparently aiming at eliciting textbook answers
some schools of economics not being represented, e.g. the Austrian school
clustering of responses according to political partisanship and departmental affiliations
Related questions that might be interesting
what has been the impact of the IGM survey?
did it influence policy makers? did it influence public discourse?
does it have an impact on the future research topics of surveyed researchers?
which fields are most promising candidates for a similar survey platform? are there already similar surveys in other fields?
what influences the buy-in that the IGM experts panel enjoys from academics?
what tradeoffs are involved in different formats of knowledge elicitation?
Research idea: Evaluate the IGM economic experts panel
Surveying scientific experts seems like a promising way to gather informed opinions relevant for important decisions, and to improve common knowledge on important topics.
The IGM economic experts panel has been doing surveys for roughly 10 years, and I wonder how well both the surveyed consensus and the format of the platform hold up, as it might be a good idea to
copy the general IGM platform to other scientific fields,
offer ideas to improve the IGM panel, and more ambitiously to improve the economics consensus (e.g. by spotting biases), and
use the survey results when they interact with topics relevant for EA.
Two ideas how to evaluate the survey’s usefulness
Look through IGM’s survey questions and evaluate those which have something like a “correct” answer in retrospect. This requires somebody fairly informed about the surveyed issues.
Interview someone from IGM, or a researcher knowledgeable about expert consensuses in economics or generally (suggestions?).
Two articles about the IGM survey
Tyler Cowen evaluating some IGM polls in 2015: “But quite seriously, my opinion of the professional consensus — on topics outside an individual’s research specialty — really has gone down as a result of these polls.” (link)
discussion of biases in the economics elite, in the surveyed IGM questions and for which experts were selected (paper from 2016)
IGM asking softball questions apparently aiming at eliciting textbook answers
some schools of economics not being represented, e.g. the Austrian school
clustering of responses according to political partisanship and departmental affiliations
Related questions that might be interesting
what has been the impact of the IGM survey?
did it influence policy makers? did it influence public discourse?
does it have an impact on the future research topics of surveyed researchers?
which fields are most promising candidates for a similar survey platform? are there already similar surveys in other fields?
what influences the buy-in that the IGM experts panel enjoys from academics?
what tradeoffs are involved in different formats of knowledge elicitation?