I think that this isnât a useful way of looking at the situation and doesnât match the reality well. I donât have time to fully elaborate on why I think that, but here are brief points:
The difference between the first and second choice applicant in terms of their fit for the role can often be quite large (in expectation)
The person who wouldâve been picked if the top ranked person didnât apply or turned the job down can still probably go do something else.
This in fact very much happens; quite often the people who nearly get an offer also get another cool offer at a similar time, and that other offer is probably a better fit for them.
Also sometimes an org will help ânear-miss candidatesâ find other opportunities (e.g., recommending them for things)
People should probably be applying to many things, which helps with this
A decent fraction of the time, an org just wonât hire someone at all if no applicant meets some high bar, or will hire an extra person if an extra person meets the high bar
Iâve observed both
My understanding is that unstructured interviews donât have much validity or reliability, but structured interviews are better, and that will almost always be only one step of a multi-step process anyway (maybe alongside 2-3 work tests, a bit of weight on a CV, and reference checks). The process as a whole still has noise and probably bias (e.g., maybe screening harder than is ideal for people with prior EA or cause-area knowledge), but itâs probably far more reliable and valid than guesswork, and reliable and valid enough to still create substantial expected differences between the impact of the top ranked people taking the role and the impact of the next ranked people taking the role.
Even âindividual contributorâ roles like researcher, as opposed to ops or manager type roles, can still help scale up an org or field and help new people get in. E.g., researchers could do distillation or disentanglement work that makes it easier for other people to get up to speed or find out where to contribute.
If youâre otherwise unusually likely to turn away from EAish stuff â i.e. reach the end of your runway or burn out â just apply. Probably even if youâre just at an average level of risk.
If you can see yourself turning down an awesome offer because you disagree with the result of the interview process, apply a bit more liberally than otherwise.
When prioritizing between positions, assign:
1000x weight to completely idiosyncratic high-impact projects (regardless of whether theyâre your ideas or someone elseâs) that no one else would otherwise pursue for a long time,
100x weight to relatively neglected roles in the community (say, because they require a rare combination of skills or because the org is new and fairly unknown),
10x to any capacity-creating kind of role to reduce the risk that they may not find anyone, and
Thank you for taking the time to write up the summary!
Possibly. Iâve only hired for two roles so far (using a structured process). In one case there were clear candidates 1, 2, and 3+, and while 2 mightâve just had a bad day, we made offers to 1 and 2 anyway. In another case, though, we had two, or possibly three, candidates tied for the top spot. Two, we thought, would be more pleasant to work with while the third one seemed to have the stronger technical skill. We didnât know how to trade that off and ended up making the offer to the one with the stronger technical skill. I have no idea whether that was the right call.
Yes, thatâs helpful for mitigating the worst-case risks. We also did that in the second case. It still seems weak though. I imagine that in most cases theyâre not able to help the other candidates very much. We werenât either afaik.
Yes, thatâs also a system Iâve encountered, and I love it! Thatâs a strong reason in my mind to apply somewhere after all. But I donât fully trust it.
Even if an organization has enough funding for this system, they may not have enough management capacity.
They may still have a hiring goal, and upon reaching it will wind down the effort they put into hiring. That frees up resources at the org at the expense of missing out on an even better candidate. The hiring process is hopefully short in comparison to the time that the person will stay at the org, so the second probably has more leverage.
Iâd be replacing .5 or .2 people, which is much better, but no where near an ops job that creates capacity.
Okay, thatâs reassuring, but see my point 1. Then again most EA interview processes (e.g., the CLR one that Stefan described in detail a few years back) are more sophisticated than ours was. A good interview process is another minor but valuable mitigation in my mind.
Good point, but these jobs are probably in higher demand than ops jobs in EA, so the counterfactual effect is milder.
I think one crux might be that if I want to dedicate the next 5â10 years of my life to something, I have a higher bar than just âWeâve taken several precautions to make it less likely that youâll have a vastly negative effect with your work.â Those precautions are invaluable of course, but there are better alternatives for applicants.
It also take a very particular kind of mental fortitude to apply for 20+ roles, and when eventually you do get an offer, to turn it down because you think youâre likely not the best candidate. That seems like such a hard decision to make, especially if the job is really awesome, fun, and high status.
I think that this isnât a useful way of looking at the situation and doesnât match the reality well. I donât have time to fully elaborate on why I think that, but here are brief points:
The difference between the first and second choice applicant in terms of their fit for the role can often be quite large (in expectation)
The person who wouldâve been picked if the top ranked person didnât apply or turned the job down can still probably go do something else.
This in fact very much happens; quite often the people who nearly get an offer also get another cool offer at a similar time, and that other offer is probably a better fit for them.
Also sometimes an org will help ânear-miss candidatesâ find other opportunities (e.g., recommending them for things)
People should probably be applying to many things, which helps with this
A decent fraction of the time, an org just wonât hire someone at all if no applicant meets some high bar, or will hire an extra person if an extra person meets the high bar
Iâve observed both
My understanding is that unstructured interviews donât have much validity or reliability, but structured interviews are better, and that will almost always be only one step of a multi-step process anyway (maybe alongside 2-3 work tests, a bit of weight on a CV, and reference checks). The process as a whole still has noise and probably bias (e.g., maybe screening harder than is ideal for people with prior EA or cause-area knowledge), but itâs probably far more reliable and valid than guesswork, and reliable and valid enough to still create substantial expected differences between the impact of the top ranked people taking the role and the impact of the next ranked people taking the role.
Even âindividual contributorâ roles like researcher, as opposed to ops or manager type roles, can still help scale up an org or field and help new people get in. E.g., researchers could do distillation or disentanglement work that makes it easier for other people to get up to speed or find out where to contribute.
An alternative decision algorithm:
If youâre otherwise unusually likely to turn away from EAish stuff â i.e. reach the end of your runway or burn out â just apply. Probably even if youâre just at an average level of risk.
If you can see yourself turning down an awesome offer because you disagree with the result of the interview process, apply a bit more liberally than otherwise.
When prioritizing between positions, assign:
1000x weight to completely idiosyncratic high-impact projects (regardless of whether theyâre your ideas or someone elseâs) that no one else would otherwise pursue for a long time,
100x weight to relatively neglected roles in the community (say, because they require a rare combination of skills or because the org is new and fairly unknown),
10x to any capacity-creating kind of role to reduce the risk that they may not find anyone, and
1x to any other role.
Thank you for taking the time to write up the summary!
Possibly. Iâve only hired for two roles so far (using a structured process). In one case there were clear candidates 1, 2, and 3+, and while 2 mightâve just had a bad day, we made offers to 1 and 2 anyway. In another case, though, we had two, or possibly three, candidates tied for the top spot. Two, we thought, would be more pleasant to work with while the third one seemed to have the stronger technical skill. We didnât know how to trade that off and ended up making the offer to the one with the stronger technical skill. I have no idea whether that was the right call.
Yes, thatâs helpful for mitigating the worst-case risks. We also did that in the second case. It still seems weak though. I imagine that in most cases theyâre not able to help the other candidates very much. We werenât either afaik.
Yes, thatâs also a system Iâve encountered, and I love it! Thatâs a strong reason in my mind to apply somewhere after all. But I donât fully trust it.
Even if an organization has enough funding for this system, they may not have enough management capacity.
They may still have a hiring goal, and upon reaching it will wind down the effort they put into hiring. That frees up resources at the org at the expense of missing out on an even better candidate. The hiring process is hopefully short in comparison to the time that the person will stay at the org, so the second probably has more leverage.
Iâd be replacing .5 or .2 people, which is much better, but no where near an ops job that creates capacity.
Okay, thatâs reassuring, but see my point 1. Then again most EA interview processes (e.g., the CLR one that Stefan described in detail a few years back) are more sophisticated than ours was. A good interview process is another minor but valuable mitigation in my mind.
Good point, but these jobs are probably in higher demand than ops jobs in EA, so the counterfactual effect is milder.
I think one crux might be that if I want to dedicate the next 5â10 years of my life to something, I have a higher bar than just âWeâve taken several precautions to make it less likely that youâll have a vastly negative effect with your work.â Those precautions are invaluable of course, but there are better alternatives for applicants.
It also take a very particular kind of mental fortitude to apply for 20+ roles, and when eventually you do get an offer, to turn it down because you think youâre likely not the best candidate. That seems like such a hard decision to make, especially if the job is really awesome, fun, and high status.