Paying candidates to complete a test task likely increases inequality, credentialism and decreases candidate quality. If you pay candidates for their time, you’re likely to accept less candidates and lower variance candidates into the test task stage. Orgs can continue to pay top candidates to complete the test task, if they believe it measurably decreases the attrition rate, but give all candidates that pass an anonymised screening bar the chance to complete a test task.
Orgs can continue to pay top candidates to complete the test task, if they believe it measurably decreases the attrition rate, but give all candidates that pass an anonymised screening bar the chance to complete a test task.
My guess is that, for many orgs, the time cost of assessing the test task is larger than the financial cost of paying candidates to complete the test task, and that significant reasons for wanting to compensate applicants are (i) a sense of justice, (ii) wanting to avoid the appearance of unreasonably demanding lots of unpaid labour from applicants, not just wanting to encourage applicants to complete the tasks[1].
So I agree that there are good reasons for wanting more people to be able to complete test tasks. But I think that doing so would potentially significantly increase costs to orgs, and that not compensating applicants would reduce costs to orgs by less than one might imagine.
I also think the justice-implications of compensating applicants are unclear (offering pay for longer tasks may make them more accessible to poorer applicants)
It takes a significant amount of time to mark a test task. But this can be fixed by just adjusting the height of the screening bar, as opposed to using credentialist and biased methods (like looking at someone’s LinkedIn profile or CV).
My guess is that, for many orgs, the time cost of assessing the test task is larger than the financial cost of paying candidates to complete the test task
This is an empirical question, and I suspect is not true. For example, it took me 10 minutes to mark each candidates 1 hour test task. So my salary would need to be 6* higher (per unit time) than the test task payment for this to be true.
I also think the justice-implications of compensating applicants are unclear (offering pay for longer tasks may make them more accessible to poorer applicants)
So my salary would need to be 6* higher (per unit time) than the test task payment for this to be true.
Strictly speaking your salary is the wrong number here. At a minimum, you want to use the cost to the org of your work, which is your salary + other costs of employing you (and I’ve seen estimates of the other costs at 50-100% of salary). In reality, the org of course values your work more highly than the amount they pay to acquire it (otherwise… why would they acquire it at that rate) so your value per hour is higher still. Keeping in mind that the pay for work tasks generally isn’t that high, it seems pretty plausible to me that the assessment cost is primarily staff time and not money.
I see a dynamic playing out here, where a user has made a falsifiable claim, I have attempted to falsify it, and you’ve attempted to deny that the claim is falsifiable at all.
I recognise it’s easy to stumble into these dynamics, but we must acknowledge that this is epistemically destructive.
Strictly speaking your salary is the wrong number here.
I don’t think we should dismiss empirical data so quickly when it’s brought to the table—that sets a bad precedent.
other costs of employing you (and I’ve seen estimates of the other costs at 50-100% of salary
I can also provide empirical data on this if that is the crux here?
Notice that we are discussing a concrete empirical data point, that represents a 600% difference, while you’ve given a theoretical upper bound of 100%. That leaves a 500% delta.
Keeping in mind that the pay for work tasks generally isn’t that high
Would you be able to provide any concrete figures here?
In reality, the org of course values your work more highly than the amount they pay to acquire it
I view pointing to opportunity cost in the abstract as essentially an appeal to ignorance.
Not to say that opportunity costs do not exist, but you’ve failed to concretise them in a way, and that makes it hard to find the truth here.
I could make similar appeals to ignorance in support of my argument, like the idea the benefit of getting a better candidate is very high, as candidate performance is fat-tailed ect. - but I believe this is similarly epistemically destructive. If I were to make a similar claim, I would likely attempt to concretise it.
I see a dynamic playing out here, where a user has made a falsifiable claim, I have attempted to falsify it, and you’ve attempted to deny that the claim is falsifiable at all.
My claim is that the org values your time at a rate that is significantly higher than the rate they pay you for it, because the cost of employment is higher than just salary and because the employer needs to value your work above its cost for them to want to hire you. I don’t see how this is unfalsifiable. Mostly you could falsify them by asking orgs how they think about the cost of staff time, though I guess some wouldn’t model it as explicitly as this.
They do mean that we’re forced to estimate the relevant threshold instead of having a precise number, but a precise wrong number isn’t better than an imprecise (closer to) correct number.
Notice that we are discussing a concrete empirical data point, that represents a 600% difference, while you’ve given a theoretical upper bound of 100%. That leaves a 500% delta.
No, if you’re comparing the cost of doing 10 minutes of work at salary X and 60 minutes of work compensated by Y, but I argue that salary X underestimates the cost of your work by a factor of 2, your salary now only needs to be more than 3 times larger than the work trial compensation, not 5 times.
When it comes to concretising “how much does employee value exceed employee costs”, it probably varies a lot from organisation to organisation. I think there are several employers in EA who believe that after a point, paying more doesn’t really get you better people. This allows their estimates of staff time to exceed employee costs by enormous margins, because there’s no mechanism to couple the two together. I think when these differences are very extreme we should be suspicious if they’re really true, but as someone who has multiple times had to compare earning to give with direct work, I’ve frequently asked an org “how much in donations would you need to prefer the money over hiring me?” and for difficult-to-hire roles they frequently say numbers dramatically larger than the salary they are offering.
This means that your argument is not going to be uniform across organisations, but I don’t know why you’d expect it to be: surely you weren’t saying that no organisation should ever pay for a test task, but only that organisations shouldn’t pay for test tasks when doing so increases their costs of assessment to the point where they choose to assess fewer people.
My expectation is that if you asked orgs about this, they would say that they already don’t choose to assess fewer people based on cost of paying them. This seems testable, and if true, it seems to me that it makes pretty much all of the other discussion irrelevant.
It takes a significant amount of time to mark a test task. But this can be fixed by just adjusting the height of the screening bar, as opposed to using credentialist and biased methods (like looking at someone’s LinkedIn profile or CV).
Whether or not to use “credentialist and biased methods (like looking at someone’s LinkedIn profile or CV)” seems orthogonal to the discussion at hand?
The key issue seems to be that if you raise the screening bar, then you would be admitting fewer applicants to the task (the opposite of the original intention).
This is an empirical question, and I suspect is not true. For example, it took me 10 minutes to mark each candidates 1 hour test task. So my salary would need to be 6* higher (per unit time) than the test task payment for this to be true.
This will definitely vary by org and by task. But many EA orgs report valuing their staff’s time extremely highly. And my impression is that both grading longer tasks and then processing the additional applicants (many orgs will also feel compelled to offer at least some feedback if a candidate has completed a multi-hour task) will often take much longer than 10 minutes total.
Paying candidates to complete a test task likely increases inequality, credentialism and decreases candidate quality. If you pay candidates for their time, you’re likely to accept less candidates and lower variance candidates into the test task stage. Orgs can continue to pay top candidates to complete the test task, if they believe it measurably decreases the attrition rate, but give all candidates that pass an anonymised screening bar the chance to complete a test task.
My guess is that, for many orgs, the time cost of assessing the test task is larger than the financial cost of paying candidates to complete the test task, and that significant reasons for wanting to compensate applicants are (i) a sense of justice, (ii) wanting to avoid the appearance of unreasonably demanding lots of unpaid labour from applicants, not just wanting to encourage applicants to complete the tasks[1].
So I agree that there are good reasons for wanting more people to be able to complete test tasks. But I think that doing so would potentially significantly increase costs to orgs, and that not compensating applicants would reduce costs to orgs by less than one might imagine.
I also think the justice-implications of compensating applicants are unclear (offering pay for longer tasks may make them more accessible to poorer applicants)
I think that many applicants are highly motivated to complete tasks, in order to have a chance of getting the job.
It takes a significant amount of time to mark a test task. But this can be fixed by just adjusting the height of the screening bar, as opposed to using credentialist and biased methods (like looking at someone’s LinkedIn profile or CV).
This is an empirical question, and I suspect is not true. For example, it took me 10 minutes to mark each candidates 1 hour test task. So my salary would need to be 6* higher (per unit time) than the test task payment for this to be true.
This is a good point.
Strictly speaking your salary is the wrong number here. At a minimum, you want to use the cost to the org of your work, which is your salary + other costs of employing you (and I’ve seen estimates of the other costs at 50-100% of salary). In reality, the org of course values your work more highly than the amount they pay to acquire it (otherwise… why would they acquire it at that rate) so your value per hour is higher still. Keeping in mind that the pay for work tasks generally isn’t that high, it seems pretty plausible to me that the assessment cost is primarily staff time and not money.
I see a dynamic playing out here, where a user has made a falsifiable claim, I have attempted to falsify it, and you’ve attempted to deny that the claim is falsifiable at all.
I recognise it’s easy to stumble into these dynamics, but we must acknowledge that this is epistemically destructive.
I don’t think we should dismiss empirical data so quickly when it’s brought to the table—that sets a bad precedent.
I can also provide empirical data on this if that is the crux here?
Notice that we are discussing a concrete empirical data point, that represents a 600% difference, while you’ve given a theoretical upper bound of 100%. That leaves a 500% delta.
Would you be able to provide any concrete figures here?
I view pointing to opportunity cost in the abstract as essentially an appeal to ignorance.
Not to say that opportunity costs do not exist, but you’ve failed to concretise them in a way, and that makes it hard to find the truth here.
I could make similar appeals to ignorance in support of my argument, like the idea the benefit of getting a better candidate is very high, as candidate performance is fat-tailed ect. - but I believe this is similarly epistemically destructive. If I were to make a similar claim, I would likely attempt to concretise it.
My claim is that the org values your time at a rate that is significantly higher than the rate they pay you for it, because the cost of employment is higher than just salary and because the employer needs to value your work above its cost for them to want to hire you. I don’t see how this is unfalsifiable. Mostly you could falsify them by asking orgs how they think about the cost of staff time, though I guess some wouldn’t model it as explicitly as this.
They do mean that we’re forced to estimate the relevant threshold instead of having a precise number, but a precise wrong number isn’t better than an imprecise (closer to) correct number.
No, if you’re comparing the cost of doing 10 minutes of work at salary X and 60 minutes of work compensated by Y, but I argue that salary X underestimates the cost of your work by a factor of 2, your salary now only needs to be more than 3 times larger than the work trial compensation, not 5 times.
When it comes to concretising “how much does employee value exceed employee costs”, it probably varies a lot from organisation to organisation. I think there are several employers in EA who believe that after a point, paying more doesn’t really get you better people. This allows their estimates of staff time to exceed employee costs by enormous margins, because there’s no mechanism to couple the two together. I think when these differences are very extreme we should be suspicious if they’re really true, but as someone who has multiple times had to compare earning to give with direct work, I’ve frequently asked an org “how much in donations would you need to prefer the money over hiring me?” and for difficult-to-hire roles they frequently say numbers dramatically larger than the salary they are offering.
This means that your argument is not going to be uniform across organisations, but I don’t know why you’d expect it to be: surely you weren’t saying that no organisation should ever pay for a test task, but only that organisations shouldn’t pay for test tasks when doing so increases their costs of assessment to the point where they choose to assess fewer people.
My expectation is that if you asked orgs about this, they would say that they already don’t choose to assess fewer people based on cost of paying them. This seems testable, and if true, it seems to me that it makes pretty much all of the other discussion irrelevant.
Whether or not to use “credentialist and biased methods (like looking at someone’s LinkedIn profile or CV)” seems orthogonal to the discussion at hand?
The key issue seems to be that if you raise the screening bar, then you would be admitting fewer applicants to the task (the opposite of the original intention).
This will definitely vary by org and by task. But many EA orgs report valuing their staff’s time extremely highly. And my impression is that both grading longer tasks and then processing the additional applicants (many orgs will also feel compelled to offer at least some feedback if a candidate has completed a multi-hour task) will often take much longer than 10 minutes total.