I’ve done a lot of partially blind hiring processes both within EA and outside it [1]. And as much as I like them (and feel like I’ve benefited from them), I think there’s good reasons why they aren’t done more.
It seems really hard to create a good blind hiring process. Most of the ones I felt good about were constructed with an immense amount of care to balance not rejecting good candidates but still having enough subtlety to distinguish candidates that would pass a final stage. Even then I still felt like there would be exceptional candidates that would have fallen through the cracks because they were bad at the specific skills being tested in the blind stage. I still think the benefits are overall positive but I’m not super confident in that given the risk of mistakenly rejecting good people.
There were always at least two blind stages before the final un-blinded one: the first being a fast filter (either an automated quiz or a short task that could be graded in 1-2 minutes) and the second being a very in-depth task. Granted, this isn’t experimental evidence but it does suggest that one fast blind part isn’t enough.
The second in-depth task seems very time-consuming to maintain. Having graded anonymous work tests for my current role where there was a big “we want to hire overlooked internationals” motivation, I felt like I needed at least 5 minutes per task before I felt even semi-confident what my final feeling was going to be and usually quite a bit more to be sure and fulfill the spirit of a blind evaluation.
Many roles have downside asymmetry. I’ve mostly seen blind tests in highly technical fields where the organization can benefit if the candidate is a superstar but also isolate the damage if they turn out to be highly toxic. With operational roles, the downside is much larger while the benefits are smaller.
Anecdotally, blind hiring these days doesn’t seem guaranteed to increase demographic diversity and may even decrease it a tiny bit. I feel the most confident on this with initiatives to increase women in software engineering via hiring practices. But I’m a lot less confident on country-of-origin. My hunch is it would backfire a bit in the non-profit world, especially in effective animal advocacy building where there seems to be some attempt to build multinational capacity.
Applied seriously to a software engineer role at a prestigious tech firm and had a final stage interview that far exceeded my abilities, which was painful for everyone involved. Applied on a whim to the Charity Entrepreneurship incubation program and got rejected after the second (third?) stage. Accepted the job offer to my current credentialed non-EA job after two blind stages and a final in-person. Applied seriously to Charity Entrepreneurship Research Training program and got rejected after second stage. Applied as a long shot to a GiveWell Senior Researcher role and got rejected after second stage.
I’ve done a lot of partially blind hiring processes both within EA and outside it [1]. And as much as I like them (and feel like I’ve benefited from them), I think there’s good reasons why they aren’t done more.
It seems really hard to create a good blind hiring process. Most of the ones I felt good about were constructed with an immense amount of care to balance not rejecting good candidates but still having enough subtlety to distinguish candidates that would pass a final stage. Even then I still felt like there would be exceptional candidates that would have fallen through the cracks because they were bad at the specific skills being tested in the blind stage. I still think the benefits are overall positive but I’m not super confident in that given the risk of mistakenly rejecting good people.
There were always at least two blind stages before the final un-blinded one: the first being a fast filter (either an automated quiz or a short task that could be graded in 1-2 minutes) and the second being a very in-depth task. Granted, this isn’t experimental evidence but it does suggest that one fast blind part isn’t enough.
The second in-depth task seems very time-consuming to maintain. Having graded anonymous work tests for my current role where there was a big “we want to hire overlooked internationals” motivation, I felt like I needed at least 5 minutes per task before I felt even semi-confident what my final feeling was going to be and usually quite a bit more to be sure and fulfill the spirit of a blind evaluation.
Many roles have downside asymmetry. I’ve mostly seen blind tests in highly technical fields where the organization can benefit if the candidate is a superstar but also isolate the damage if they turn out to be highly toxic. With operational roles, the downside is much larger while the benefits are smaller.
Anecdotally, blind hiring these days doesn’t seem guaranteed to increase demographic diversity and may even decrease it a tiny bit. I feel the most confident on this with initiatives to increase women in software engineering via hiring practices. But I’m a lot less confident on country-of-origin. My hunch is it would backfire a bit in the non-profit world, especially in effective animal advocacy building where there seems to be some attempt to build multinational capacity.
Applied seriously to a software engineer role at a prestigious tech firm and had a final stage interview that far exceeded my abilities, which was painful for everyone involved. Applied on a whim to the Charity Entrepreneurship incubation program and got rejected after the second (third?) stage. Accepted the job offer to my current credentialed non-EA job after two blind stages and a final in-person. Applied seriously to Charity Entrepreneurship Research Training program and got rejected after second stage. Applied as a long shot to a GiveWell Senior Researcher role and got rejected after second stage.