Definitely agree on “should,” assuming it’s tractable. As for “can”, one possible approach is to hunt down the references in Hunter and Schimdt[1], or similar/more recent meta-analyses, disaggregate by career fields that are interesting to EAs, and look at what specific questions are asked in things like “work sample tests” and “structured employment interviews.”
Ideally you want questions that are a) predictive, b)relatively uncorrelated with general mental ability[2] and c) are reasonable to ask earlier on in someone’s studies[3].
One reason to be cynical of this approach is that personnel selection is a well-researched and economically really lucrative if for-profit companies can figure it out, and yet very good methods do not already exist.
One reason to be optimistic is that if we’re trying to help EAs figure out their own personal skills/comparative advantages, this is less subject to adversarial effects.
Definitely agree on “should,” assuming it’s tractable. As for “can”, one possible approach is to hunt down the references in Hunter and Schimdt[1], or similar/more recent meta-analyses, disaggregate by career fields that are interesting to EAs, and look at what specific questions are asked in things like “work sample tests” and “structured employment interviews.”
Ideally you want questions that are a) predictive, b)relatively uncorrelated with general mental ability[2] and c) are reasonable to ask earlier on in someone’s studies[3].
One reason to be cynical of this approach is that personnel selection is a well-researched and economically really lucrative if for-profit companies can figure it out, and yet very good methods do not already exist.
One reason to be optimistic is that if we’re trying to help EAs figure out their own personal skills/comparative advantages, this is less subject to adversarial effects.
[1] https://pdfs.semanticscholar.org/8f5c/b88eed2c3e9bd134b46b14b6103ebf41c93e.pdf
[2] Because if the question just tests how smart you are, it says something about absolute advantage but not comparative.
[3] Otherwise this will ruin the point of cheap tests.