I agree with a lot in this post! Especially doing work tests for high-skilled roles, and getting information from people with conflicts of interest. Thanks for writing this!
Run unstructured interviews
I’m not sure I agree with the straightforward reading of this (though maybe that’s slightly different than what you mean). I think the case for running semi-structured interviews is better than running structured ones. But overall, I don’t take “structured interviews” to mean “only ask the same set of questions”. I take at least some of the literature on it to refer to asking roughly the same questions, then using structured follow ups until you’ve gotten a lot of information on each question from a candidate. The important part of the “structure” to me is trying to get comparability on their skills on specific attributes, not comparability on their response to the question.
Some general reflections I’ve had about hiring that feel related to this, but also make me skeptical of deviating from best practice (e.g. develop a job analysis, test people directly on the items in the job analysis in a structured way).
The more hiring I do, the less confidence I feel about ~any practice. This pushes me more and more toward “see how they do at the actual job or the skills that seem needed for the job.”
I think that often what I want to do in hiring is basically very biased by what’s pleasant for me. Structured interviews are very boring, especially when you do a lot of them. It feels like I’m not getting to know someone deeply. But when I look back at my most successful hires, it’s almost always me following a well designed process. The only cases I see where I deviated heavily from the process and it went well was when I had worked extensively with the person before and knew them quite well (though this could just be evidence that I’m not very good at casually assessing people without structure).
Relatedly, this is anecdotal, but I notice organizations who are less formal in their hiring seem also less happy with their hires. I don’t feel as confident in this claim though.
Hiring is also hard if you’re progressively eliminating candidates across rounds, because you never can measure the candidates you rejected. The candidate pool is always biased by who you chose to advance already. This makes me feel like I’m never collecting particularly useful data on hiring in hiring rounds. I don’t ever learn how good the people I rejected were!
Informal references are useful
I agree that these are very useful, but I’ve also noticed them increasing in frequency in the ecosystem, and that concerns me. I think these can be really biasing. I’ve started turning down most requests for these for a few reasons, and now usually ask the hiring manager if they can get the candidate’s consent for my reference.
If someone left a workplace in a termination, sometimes, especially with employers of record, there might be a non-disparagement clause. My choice to take or turndown an informal reference might be primarily informed by my legal obligations, but because I can’t say that, the person collecting the reference doesn’t have any info on why I’m saying I won’t give a reference.
I often feel like informal references don’t pass a smell test of “if the candidate knew the hiring manager was collecting this, would they be okay with it.” That feels like I’m breaking a candidate’s trust.
I’ve been asked for informal references for my direct reports. I’m thankful that often these direct reports have shared that they were job searching with me, but if they hadn’t, they probably would rightfully consider this a pretty big violation of trust on the hiring manager’s end.
There are cases where the above don’t apply, but I do think candidates deserve to know that there is some level of reference collecting happening from references they didn’t share.
The only cases I see where I deviated heavily from the process and it went well was when I had worked extensively with the person before and knew them quite well
Do you think this basically supports “hiring people you know to be good” and using your network and previous interactions with people not in an interview setting to seek out good candidates?
Yeah, I think it provides some evidence in favor of it, but there are lots of downsides to that too, like:
Obviously, there is risk a bias, etc. (e.g. I know a small subset of possible people!)
Lots of times, I don’t actually know someone who would be a particularly good fit.
I think that doing this is lower downside risk, but probably somewhat lower upside potential in expectation, and probably just varies case-to-case in how those shake out overall.
Hiring is also hard if you’re progressively eliminating candidates across rounds, because you never can measure the candidates you rejected. The candidate pool is always biased by who you chose to advance already. This makes me feel like I’m never collecting particularly useful data on hiring in hiring rounds. I don’t ever learn how good the people I rejected were!
Isn’t this avoidable? I could imagine a system where you allow a small percentage of randomized “rejected” candidates to the next hiring round and, if they properly succeed in the next, allow them into the third round. I have essentially no experience with how hiring works, but it seems to me that this could i) increase the effort that goes into hiring only moderately, ii) still sounds kind of fair to the candidates, iii) and would give you some information on what your selection process actually selects for.
Yeah definitely, I think that would be a really reasonable thing to do, and is the kind of experimentation I want to see in hiring in the space that I talk about here!
I don’t ever learn how good the people I rejected were!
Definitely agree this is a challenge and limits learning from hiring rounds. I don’t quite agree with the strength of this statement though. A great thing about the agency and tenacity of people in our community is that people often apply to a bunch of roles at similar organisations, and so you end up seeing what people you turned down work on from afar, and how well that goes. It seems reasonably informative to me to track which people end up doing great work at other orgs, and thinking through how much of that seems down to the role being different vs your hiring process having been a false negative, and thinking through what the process would have looked like to have avoided the false negative.
I agree with a lot in this post! Especially doing work tests for high-skilled roles, and getting information from people with conflicts of interest. Thanks for writing this!
I’m not sure I agree with the straightforward reading of this (though maybe that’s slightly different than what you mean). I think the case for running semi-structured interviews is better than running structured ones. But overall, I don’t take “structured interviews” to mean “only ask the same set of questions”. I take at least some of the literature on it to refer to asking roughly the same questions, then using structured follow ups until you’ve gotten a lot of information on each question from a candidate. The important part of the “structure” to me is trying to get comparability on their skills on specific attributes, not comparability on their response to the question.
Some general reflections I’ve had about hiring that feel related to this, but also make me skeptical of deviating from best practice (e.g. develop a job analysis, test people directly on the items in the job analysis in a structured way).
The more hiring I do, the less confidence I feel about ~any practice. This pushes me more and more toward “see how they do at the actual job or the skills that seem needed for the job.”
I think that often what I want to do in hiring is basically very biased by what’s pleasant for me. Structured interviews are very boring, especially when you do a lot of them. It feels like I’m not getting to know someone deeply. But when I look back at my most successful hires, it’s almost always me following a well designed process. The only cases I see where I deviated heavily from the process and it went well was when I had worked extensively with the person before and knew them quite well (though this could just be evidence that I’m not very good at casually assessing people without structure).
Relatedly, this is anecdotal, but I notice organizations who are less formal in their hiring seem also less happy with their hires. I don’t feel as confident in this claim though.
Hiring is also hard if you’re progressively eliminating candidates across rounds, because you never can measure the candidates you rejected. The candidate pool is always biased by who you chose to advance already. This makes me feel like I’m never collecting particularly useful data on hiring in hiring rounds. I don’t ever learn how good the people I rejected were!
I agree that these are very useful, but I’ve also noticed them increasing in frequency in the ecosystem, and that concerns me. I think these can be really biasing. I’ve started turning down most requests for these for a few reasons, and now usually ask the hiring manager if they can get the candidate’s consent for my reference.
If someone left a workplace in a termination, sometimes, especially with employers of record, there might be a non-disparagement clause. My choice to take or turndown an informal reference might be primarily informed by my legal obligations, but because I can’t say that, the person collecting the reference doesn’t have any info on why I’m saying I won’t give a reference.
I often feel like informal references don’t pass a smell test of “if the candidate knew the hiring manager was collecting this, would they be okay with it.” That feels like I’m breaking a candidate’s trust.
I’ve been asked for informal references for my direct reports. I’m thankful that often these direct reports have shared that they were job searching with me, but if they hadn’t, they probably would rightfully consider this a pretty big violation of trust on the hiring manager’s end.
There are cases where the above don’t apply, but I do think candidates deserve to know that there is some level of reference collecting happening from references they didn’t share.
Nice comment. One follow-up.
Do you think this basically supports “hiring people you know to be good” and using your network and previous interactions with people not in an interview setting to seek out good candidates?
Yeah, I think it provides some evidence in favor of it, but there are lots of downsides to that too, like:
Obviously, there is risk a bias, etc. (e.g. I know a small subset of possible people!)
Lots of times, I don’t actually know someone who would be a particularly good fit.
I think that doing this is lower downside risk, but probably somewhat lower upside potential in expectation, and probably just varies case-to-case in how those shake out overall.
Isn’t this avoidable? I could imagine a system where you allow a small percentage of randomized “rejected” candidates to the next hiring round and, if they properly succeed in the next, allow them into the third round. I have essentially no experience with how hiring works, but it seems to me that this could i) increase the effort that goes into hiring only moderately, ii) still sounds kind of fair to the candidates, iii) and would give you some information on what your selection process actually selects for.
Yeah definitely, I think that would be a really reasonable thing to do, and is the kind of experimentation I want to see in hiring in the space that I talk about here!
Definitely agree this is a challenge and limits learning from hiring rounds. I don’t quite agree with the strength of this statement though. A great thing about the agency and tenacity of people in our community is that people often apply to a bunch of roles at similar organisations, and so you end up seeing what people you turned down work on from afar, and how well that goes. It seems reasonably informative to me to track which people end up doing great work at other orgs, and thinking through how much of that seems down to the role being different vs your hiring process having been a false negative, and thinking through what the process would have looked like to have avoided the false negative.