Not Peter, but looking at the last ~20 roles I’ve hired for, I’d guess that during hiring, maybe 15 or so had an alternative candidate who seemed worth hiring (though perhaps did worse in some scoring system). These were all operations roles within an EA organization. For 2 more senior roles I hired for during that time, there appeared to be suitable alternatives. For other less senior roles there weren’t (though I think the opposite generally tends to be more true).
I do thing one consideration here is we are talking about who looked best during hiring. That’s different than who would be a better employee—we’re assuming our hiring process does a good job of assessing people’s fit / job performance, etc., and we know that the best predictors during hiring are only moderately correlated with later job performance, so it’s plausible that often we think there is a big gap between two candidates, but they’d actually perform equally well (or that someone who seems like the best candidate isn’t). Hiring is just a highly uncertain business, and predicting long-term job performance from like, 10 hours of sample work and interviews is pretty hard — I’m somewhat skeptical that looking at hiring data is even the right approach, because you’d also want to control for things like if those employees always meet performance expectations in the future, etc, and you never actually get counterfactual data on how good the person you didn’t hire was. I’m certain that many EA organizations have hired someone who appeared to be better than the alternative by a wide margin, and easily cleared a hiring bar, but who later turned out to have major performance issues, even if the organization was doing a really good job evaluating people.
Not Peter, but looking at the last ~20 roles I’ve hired for, I’d guess that during hiring, maybe 15 or so had an alternative candidate who seemed worth hiring (though perhaps did worse in some scoring system). These were all operations roles within an EA organization. For 2 more senior roles I hired for during that time, there appeared to be suitable alternatives. For other less senior roles there weren’t (though I think the opposite generally tends to be more true).
I do thing one consideration here is we are talking about who looked best during hiring. That’s different than who would be a better employee—we’re assuming our hiring process does a good job of assessing people’s fit / job performance, etc., and we know that the best predictors during hiring are only moderately correlated with later job performance, so it’s plausible that often we think there is a big gap between two candidates, but they’d actually perform equally well (or that someone who seems like the best candidate isn’t). Hiring is just a highly uncertain business, and predicting long-term job performance from like, 10 hours of sample work and interviews is pretty hard — I’m somewhat skeptical that looking at hiring data is even the right approach, because you’d also want to control for things like if those employees always meet performance expectations in the future, etc, and you never actually get counterfactual data on how good the person you didn’t hire was. I’m certain that many EA organizations have hired someone who appeared to be better than the alternative by a wide margin, and easily cleared a hiring bar, but who later turned out to have major performance issues, even if the organization was doing a really good job evaluating people.