Brilliant post. I have been reading about and dabbling in evidence-led HR for some time, and on my to-do list was writing this exact post for the forum (although I suspect mine would’ve likely been of a significantly lower quality, so I’m not mad at all that you beat me to it)
How can an organization effectively use intelligence tests & personality tests in hiring, while avoiding/minimizing legal risks? I know that many are rubbish, but are they all varying degrees of rubbish, or are some of them noticeably better than the rest? How good is Wonderlic? The gap between this can help you select better applicants and here are the details of how to implement this seem fairly large. What research has been done in the recent two decades?
I didn’t see you cite it, but I assume you’ve come across Schmidt’s 2016 update to his ’85 years’ paper? See here.
I do think that such a quiz should be used with caution, as it can make the hiring process feel fairly impersonal for the applicant. Depending on what population you want to recruit from, some applicants may strongly prefer human contact rather than to use a quiz, so much then when prompted to do the quiz they choose to drop out of the application process.
I wasn’t aware of that China-specific example you outline in the footnote, so it’s a nice reminder that cross-cultural considerations are important and often surprising. However, that aside, in general I sense that it’s a win-win if applicants drop out of the process because they don’t like the initial quiz-based filtering process. Self-selection of this kind saves time on both ends. While candidates might think it impersonal, I have a strong prior against using that as a reason to change the initial screening to something less predictive like CV screening.
perhaps I’ll find answers to some of these questions during the coming months as I continue to work my way through my to-read list
Would be curious to see this list, for my own purposes :)
For some jobs, it is easy to create work sample tests, but for a lot of knowledge work jobs it is hard to think up what a good work sample test could be. I’d love to see some kind of a work sample test example book, in which example work samples for many different jobs are written out. Does something like this exist?
How to create good structured interviews? The gap between this is more effective than unstructured interviews and here are the details of how to implement this seem fairly large. Are there resources that go more in-depth than re:Work?
I often hear people asking for resources like this. Like you, I suspect that the significant effort investment required in closing the implementation gap is what keeps some people form implementing more predictive selection methods.
I’m not sure what your MO is, but I’d be keen to collaborate on future posts you’re thinking of writing or even putting together some resources for the EA community on some of the above (e.g. example work samples, index of General Mental Ability tests, etc). Won’t be offended if you prefer to work solo; we can always stay in touch to ensure we’re not both going to publish the same thing at the same time :)
I’d be keen to collaborate on future posts you’re thinking of writing or even putting together some resources for the EA community on some of the above
I’d be open to that. I don’t have any posts like that on my docket at the moment, but I’ll keep it in mind. If nothing else, I’d be happy for you to share a Google Doc link with me and I can use suggesting mode to make edits on a draft of yours.
I assume you’ve come across Schmidt’s 2016 update to his ’85 years’ paper?
I haven’t come across it, but this is just the kind of thing I was hoping someone would refer me to! Thank you very much. I’ve had many concerns about The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings, mainly about the data used, and I am very happy to see an update.
Brilliant post. I have been reading about and dabbling in evidence-led HR for some time, and on my to-do list was writing this exact post for the forum (although I suspect mine would’ve likely been of a significantly lower quality, so I’m not mad at all that you beat me to it)
I didn’t see you cite it, but I assume you’ve come across Schmidt’s 2016 update to his ’85 years’ paper? See here.
I wasn’t aware of that China-specific example you outline in the footnote, so it’s a nice reminder that cross-cultural considerations are important and often surprising. However, that aside, in general I sense that it’s a win-win if applicants drop out of the process because they don’t like the initial quiz-based filtering process. Self-selection of this kind saves time on both ends. While candidates might think it impersonal, I have a strong prior against using that as a reason to change the initial screening to something less predictive like CV screening.
Would be curious to see this list, for my own purposes :)
I often hear people asking for resources like this. Like you, I suspect that the significant effort investment required in closing the implementation gap is what keeps some people form implementing more predictive selection methods.
I’m not sure what your MO is, but I’d be keen to collaborate on future posts you’re thinking of writing or even putting together some resources for the EA community on some of the above (e.g. example work samples, index of General Mental Ability tests, etc). Won’t be offended if you prefer to work solo; we can always stay in touch to ensure we’re not both going to publish the same thing at the same time :)
I’d be open to that. I don’t have any posts like that on my docket at the moment, but I’ll keep it in mind. If nothing else, I’d be happy for you to share a Google Doc link with me and I can use suggesting mode to make edits on a draft of yours.
I haven’t come across it, but this is just the kind of thing I was hoping someone would refer me to! Thank you very much. I’ve had many concerns about The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings, mainly about the data used, and I am very happy to see an update.