Can I check my understanding here? It sounds like:
In a hiring process, later screens are less correlated with applicant future performance than earlier screens (because in general, applicants who do well in your first form screen will do more similarly in the interviews than your applicant pool overall). This does seem like an interesting phenomenon that I hadn’t thought about before, thanks :)
I guess I’m confused what hiring managers are supposed to do about this effect, though. I skimmed the chapter you link but it seemed more relevant for correcting for this when e.g. trying to correct for this when measuring the effectiveness of different interview tools in the wild.
Would be curious for your off the cuff thoughts (but this is mostly just for my curiosity so no worries if you’re busy :) )
I think you are roughly correct: the chapter is mainly about using statistics to correct for range restriction.
My interpretation/takeaway (and my memory from reading this paper a while back) is that many selection methods are less predictive than we think due to using a pre-selected sample (the people that passed some other screen/test). I wish I had some easy actions that a hiring manager could take, but I’m afraid that I don’t. I only have vague concepts, like “be more humble about your methods” and “try not to have too much confidence that you have a good system,” which are not very actionable.
I’ve barely started to touch this type of topic and consider myself quite a novice, but there are some people in/connected EA that (I’m guessing) know more about it than I do if you ever want to get more context on it within an EA context. I’m guessing that Bret at Charity Entrepreneurship and Veronica at Rethink Priorities both know more about range restriction than I do, and would have more context of the research and how to apply it. You could also reach out to Brennan Wilkerson; he studied some related stuff as part of his master’s degree in Management. Let me know if you want intros to any of these people.
Can I check my understanding here? It sounds like:
In a hiring process, later screens are less correlated with applicant future performance than earlier screens (because in general, applicants who do well in your first form screen will do more similarly in the interviews than your applicant pool overall). This does seem like an interesting phenomenon that I hadn’t thought about before, thanks :)
I guess I’m confused what hiring managers are supposed to do about this effect, though. I skimmed the chapter you link but it seemed more relevant for correcting for this when e.g. trying to correct for this when measuring the effectiveness of different interview tools in the wild.
Would be curious for your off the cuff thoughts (but this is mostly just for my curiosity so no worries if you’re busy :) )
I think you are roughly correct: the chapter is mainly about using statistics to correct for range restriction.
My interpretation/takeaway (and my memory from reading this paper a while back) is that many selection methods are less predictive than we think due to using a pre-selected sample (the people that passed some other screen/test). I wish I had some easy actions that a hiring manager could take, but I’m afraid that I don’t. I only have vague concepts, like “be more humble about your methods” and “try not to have too much confidence that you have a good system,” which are not very actionable.
I’ve barely started to touch this type of topic and consider myself quite a novice, but there are some people in/connected EA that (I’m guessing) know more about it than I do if you ever want to get more context on it within an EA context. I’m guessing that Bret at Charity Entrepreneurship and Veronica at Rethink Priorities both know more about range restriction than I do, and would have more context of the research and how to apply it. You could also reach out to Brennan Wilkerson; he studied some related stuff as part of his master’s degree in Management. Let me know if you want intros to any of these people.