Hi everyone thank you for your time and thoughts here.
Can you weigh in on work tests? I have done ~4 work tests now, and a role was posted that I was excited about that required a 7 hour (!) paid work test, and it’s really diminished my motivation to apply there. I have my own opinions as a 15 year professional, but would love to get other views—both from hiring managers and job seekers. Full disclosure.… that 7 hour thing is hot on my mind and doesn’t represent the typical length I’ve seen (more like 4). Also I value work tests—for both sides—everything below is about course correction for efficient interview effort vs hiring satisfaction, not direction changing away from work tests.
1 − 80k’s book cites a big study where correlation between selection test and job performance is 0.54 (1st place) for Work Sample Tests, and 0.51 (3rd place) for Interviews (structured) - IQ test is 2nd. Not a huge gap, and neither very impressive overall. I understand prioritizing the best, but it seems an immense amount of priority is placed on tests vs interviews. Both employer and candidate put a ton of collective work into work test sampling—creating, doing, and assessing.
2 - I appreciate the payment that comes with most tests. I am lucky enough to be a) fully employed and b) not care too much about work test payment, and care more about my time. But 7 hours.. I’m kind of like jeesh… If you have a full time job (me) or a family (me) or just a lot going on in your life (everyone), 7 hours is two free nights, or a whole weekend day. It just seems overkill as a screening method, or even after a single screening call which is mostly what I see.
3 - I’ve never experienced work tests outside the impact space, but I also know that my (corporate non-impact) org which does a great job recruiting capable folks also does not use them—instead a combination of personality tests (which I have yet to see in the impact space) and interviews. Not saying it wouldn’t improve with work tests being added in, but it seems like it can be effective without.
To me—emotionally—it just feels like some orgs are taking work test duration too far and dissuading capable folks from applying. Maybe it’s meant as a scarecrow for the uncommitted. It has a flavor to me of SAT-testing elitism a little bit (which I admit isn’t totally fair—these aren’t standardized tests you can hire a tutor for) where it’s a signal of seriousness. Could there be a university/academia-testing bias, given the impact space’s educated demo? Or do experienced hiring managers really see efficient value in beefing up work tests like this? Or is there some unique challenge going on in impact work that requires this kind of selection? Is this different when it’s an internal hiring team vs. consultant-driven? What about those of you that hire without work tests?
This may read as complaining, but I imagine there’s a good case for cutting way down on work test duration and getting equivalent performance results out of your selected hire for less effort on everyone’s part. Love to hear your thoughts!
Thanks for your thoughts and questions, Andrew! I’ve not addressed everything you touched on, but here are my quick takes:
I wouldn’t say an immense amount of priority is placed on work tests over interviews, at least at OP, though we do find them to be an extremely useful part of our decision-making process. It’s true that a lot of time and effort goes into creating, completing, and assessing them, but a lot of time and effort also goes into interviews, e.g. in preparation, conducting multiple rounds, taking notes, in team discussions, etc. Maybe what you’re getting at is that hiring processes are disproportionately front-loaded with WTs—which is fair!
I totally hear you on the lengthy work test bit—7 hours is a huge commitment, especially for people with full-time jobs or a family or simply a life to live (which as you say, includes everyone). I think there’s probably a point of diminishing returns with work test length, but I imagine it’s different for each depending on the signal sought: maybe one type of role doesn’t need a WT that’s longer than 2 hours, but some others are so nuanced that the point of diminishing returns doesn’t hit until 7-8 hours. This isn’t hard for me to believe, but to your point I think it’s probably right for orgs that administer WTs to always be asking, “Can we make this shorter while still capturing the signal we want to capture?” It’s a good push!
I definitely agree that a hiring process can be effective and successful without work tests. However, I think they make hiring even more successful, and the tradeoffs (e.g. the time it takes us to develop and assess them, the risk of putting off candidates who might be strong in the role) feel acceptable. Whether or not there’s an academic bias in impact orgs’ liking of WTs, I can say that hiring managers at OP, at least, find great value in the data points provided by longer WTs (otherwise, we’d retire them!).
Appreciate your response and insider view, Evan! Certainly some nuances to consider here, and I’ll keep your points in mind when I’m applying for these kinds of roles.
I also wonder whether we could do some kind of standardized test once a year on things that EA orgs uniquely care about, like reasoning transparency or understanding of EA, and then various EA orgs could use that for their hiring.
If requires some initial coordination & investment, but after that it could save both orgs and applicants quite some time and money.
I’ve heard this many times, and I feel like some orgs might indeed overdo it with work tests. I’d be curious how much more info recruiters get from 4-8h work test, compared to 1-2h + a (short) interview, and maybe allowing people to submit an existing work sample or so.
I’m glad to hear it resonates with you and others!
Right agreed—like hypothetically if you plotted a curve of [work test duration] and [employer/employee satisfaction ~6 months after the hire], where would you start to see that curve level off? I would bet it’d be a lot earlier than 7 hours (as long as hiring is still based on interviews etc)!
To your other comment about standardized tests, I think that’s interesting. I don’t know too much about where/when standardized tests are most effective when it comes to assessment in the working adult population. On the one hand, that would be pretty broad coordination with all the organizations, and there is such a broad skillset requirement dependent on roles. I think we’d know it’s valuable if hiring orgs didn’t do something in their hiring process, now that they had that assessment result for each candidate. But on the other hand, maybe there are some fundamentals like the ones you mention that could be a shorter path to screening. Another question for the recruiting folks!
Hi everyone thank you for your time and thoughts here.
Can you weigh in on work tests? I have done ~4 work tests now, and a role was posted that I was excited about that required a 7 hour (!) paid work test, and it’s really diminished my motivation to apply there. I have my own opinions as a 15 year professional, but would love to get other views—both from hiring managers and job seekers. Full disclosure.… that 7 hour thing is hot on my mind and doesn’t represent the typical length I’ve seen (more like 4). Also I value work tests—for both sides—everything below is about course correction for efficient interview effort vs hiring satisfaction, not direction changing away from work tests.
1 − 80k’s book cites a big study where correlation between selection test and job performance is 0.54 (1st place) for Work Sample Tests, and 0.51 (3rd place) for Interviews (structured) - IQ test is 2nd. Not a huge gap, and neither very impressive overall. I understand prioritizing the best, but it seems an immense amount of priority is placed on tests vs interviews. Both employer and candidate put a ton of collective work into work test sampling—creating, doing, and assessing.
2 - I appreciate the payment that comes with most tests. I am lucky enough to be a) fully employed and b) not care too much about work test payment, and care more about my time. But 7 hours.. I’m kind of like jeesh… If you have a full time job (me) or a family (me) or just a lot going on in your life (everyone), 7 hours is two free nights, or a whole weekend day. It just seems overkill as a screening method, or even after a single screening call which is mostly what I see.
3 - I’ve never experienced work tests outside the impact space, but I also know that my (corporate non-impact) org which does a great job recruiting capable folks also does not use them—instead a combination of personality tests (which I have yet to see in the impact space) and interviews. Not saying it wouldn’t improve with work tests being added in, but it seems like it can be effective without.
To me—emotionally—it just feels like some orgs are taking work test duration too far and dissuading capable folks from applying. Maybe it’s meant as a scarecrow for the uncommitted. It has a flavor to me of SAT-testing elitism a little bit (which I admit isn’t totally fair—these aren’t standardized tests you can hire a tutor for) where it’s a signal of seriousness. Could there be a university/academia-testing bias, given the impact space’s educated demo? Or do experienced hiring managers really see efficient value in beefing up work tests like this? Or is there some unique challenge going on in impact work that requires this kind of selection? Is this different when it’s an internal hiring team vs. consultant-driven? What about those of you that hire without work tests?
This may read as complaining, but I imagine there’s a good case for cutting way down on work test duration and getting equivalent performance results out of your selected hire for less effort on everyone’s part. Love to hear your thoughts!
Thanks for your thoughts and questions, Andrew! I’ve not addressed everything you touched on, but here are my quick takes:
I wouldn’t say an immense amount of priority is placed on work tests over interviews, at least at OP, though we do find them to be an extremely useful part of our decision-making process. It’s true that a lot of time and effort goes into creating, completing, and assessing them, but a lot of time and effort also goes into interviews, e.g. in preparation, conducting multiple rounds, taking notes, in team discussions, etc. Maybe what you’re getting at is that hiring processes are disproportionately front-loaded with WTs—which is fair!
I totally hear you on the lengthy work test bit—7 hours is a huge commitment, especially for people with full-time jobs or a family or simply a life to live (which as you say, includes everyone). I think there’s probably a point of diminishing returns with work test length, but I imagine it’s different for each depending on the signal sought: maybe one type of role doesn’t need a WT that’s longer than 2 hours, but some others are so nuanced that the point of diminishing returns doesn’t hit until 7-8 hours. This isn’t hard for me to believe, but to your point I think it’s probably right for orgs that administer WTs to always be asking, “Can we make this shorter while still capturing the signal we want to capture?” It’s a good push!
I definitely agree that a hiring process can be effective and successful without work tests. However, I think they make hiring even more successful, and the tradeoffs (e.g. the time it takes us to develop and assess them, the risk of putting off candidates who might be strong in the role) feel acceptable. Whether or not there’s an academic bias in impact orgs’ liking of WTs, I can say that hiring managers at OP, at least, find great value in the data points provided by longer WTs (otherwise, we’d retire them!).
Appreciate your response and insider view, Evan! Certainly some nuances to consider here, and I’ll keep your points in mind when I’m applying for these kinds of roles.
I also wonder whether we could do some kind of standardized test once a year on things that EA orgs uniquely care about, like reasoning transparency or understanding of EA, and then various EA orgs could use that for their hiring.
If requires some initial coordination & investment, but after that it could save both orgs and applicants quite some time and money.
Strongly upvoted, thanks!
I’ve heard this many times, and I feel like some orgs might indeed overdo it with work tests. I’d be curious how much more info recruiters get from 4-8h work test, compared to 1-2h + a (short) interview, and maybe allowing people to submit an existing work sample or so.
I’m glad to hear it resonates with you and others!
Right agreed—like hypothetically if you plotted a curve of [work test duration] and [employer/employee satisfaction ~6 months after the hire], where would you start to see that curve level off? I would bet it’d be a lot earlier than 7 hours (as long as hiring is still based on interviews etc)!
To your other comment about standardized tests, I think that’s interesting. I don’t know too much about where/when standardized tests are most effective when it comes to assessment in the working adult population. On the one hand, that would be pretty broad coordination with all the organizations, and there is such a broad skillset requirement dependent on roles. I think we’d know it’s valuable if hiring orgs didn’t do something in their hiring process, now that they had that assessment result for each candidate. But on the other hand, maybe there are some fundamentals like the ones you mention that could be a shorter path to screening. Another question for the recruiting folks!