This is a sloppy rough draft that I have had sitting in a Google doc for months, and I figured that if I don’t share it now, it will sit there forever. So please read this as a rough grouping of some brainstormy ideas, rather than as some sort of highly confident and well-polished thesis.
- - - - - -
What feedback do rejected applicants want?
From speaking with rejected job applicants within the EA ecosystem during the past year, I roughly conclude that they want feedback in two different ways:
The first way is just emotional care, which is really just a different way of saying “be kind rather than being mean or being neutral.”[1] They don’t want to feel bad, because rejection isn’t fun. Anybody who has been excluded from a group of friends, or kicked out of a company, or in any way excluded from something that you want to be included in knows that it can feel bad.[2] It feels even worse if you appear to meet the requirements of the job, put in time and effort to try really hard, care a lot about the community and the mission, perceive this as one of only a few paths available to you for more/higher impact, and then you get summarily excluded with a formulaic email template. There isn’t any feasible way to make a rejection feel great, but you can minimize how crappy it feels. Thank the candidates for their time/effort, and emphasize that you are rejecting this application for this role rather than rejecting this person in general. Don’t reject people immediately after their submission; wait a couple of days. If Alice submits a work trial task and less than 24 hours later you reject her, it feels to her like you barely glanced at her work, even if you spent several hours diligently going over it.
Improving. People want actionable feedback. If they lack a particular skill, they would like to know how to get better so that they can go learn that skill and then be a stronger candidate for this type of role in the future. If the main differentiator between candidates Alice and Bob is that Alice scored 50 points better on an IQ test or that Alice attended Impressive School while Bob attended No Name School, maybe don’t tell Bob that.[3] But if the main differentiator is that Alice has spent a year being a volunteer for the EA Virtual Program or that Alice is really good with spreadsheets or that Bob didn’t format his documents well, that is actionable, and gives the candidate a signal regarding how they can improve. Now the candidate knows something they can do to make become a more competitive candidate. They will practice their Excel skills and look up spreadsheet tutorials, they can get some volunteering experience with a relevant organization, and they look up how to use headers and to adjust line spacing. Think of this like a company investing in the local community college and sponsoring a professorship at the college: they are building a pipeline of potential future employees.
Here is a rough hierarchy of what, in an ideal world, I’d like to receive when I am rejected from a job application:
“Thanks for applying. We won’t be moving forward with your application. Although it is never fun to receive an email like this, we want to express appreciation for the time you spent on this selection process. Regarding why we choose to not move forward with your application, it looks like you don’t have as much experience directly related to X as the candidates we are moving forward with, and we also want someone who is able to Y. Getting experience with Y is challenging, but some ideas are here: [LINK].”
“Thanks for applying. We won’t be moving forward with your application. It looks like you don’t have as much experience directly related to X as the most competitive candidates, and we also want someone who is able to Y.”
“Thanks for applying. We won’t be moving forward with your application.”
That last bullet point is what most EA organizations send (according to conversations I’ve had with candidates, as well as my own experiences in EA hiring rounds). I have seen two or three that sometimes send rejections that are similar to the first or similar to the second.[4] If the first bullet point looks too challenging and you think that it would take too much staff time, then see if you can do the second bullet point: simply telling people why (although this will dependent on the context) can make rejections a lot less hurtful, and also points them in the right direction for how to get better.
I still remember how bad it felt being told that I couldn’t join a feminist reading group because they didn’t want any men there. I think that was totally understandable, but it still felt bad to be excluded. I remember not being able to join a professional networking group because I was older than the cutoff age (they required new members to be under 30, and I was 31 when I learned about it). These things happened years ago, and were not particularly influential in my life. But people remember being excluded.
Things that people cannot change with a reasonable amount of time and effort (or things that would require a time machine, such as what university someone attended) are generally not good pieces of feedback to give people. These things aren’t actionable.
Last I saw, the Centre for Effective Altruism and Animal Advocacy Careers both had systems in place helping them to do better than average. It has been a while since I’ve interaction with the internals of either of their hiring systems, but last I checked they both send useful and actionable feedback for at least some of their rejections.
I’m on board with a lot of your emotional care advice, but,,,
Don’t reject people immediately after their submission; wait a couple of days. If Alice submits a work trial task and less than 24 hours later you reject her, it feels to her like you barely glanced at her work, even if you spent several hours diligently going over it.
...I feel like your mileage may vary on this one. I don’t like being in suspense, and moreover it’s helpful from a planning perspective to know what’s up sooner rather than later. I’d say instead that if you want to signal that you spent time with someone’s application, do it by making sure your rejection is conspicuously specific (i.e. mentions features of the applicant or their submissions, even if only superficially).
I also think you missed an entire third category of reason to want feedback, which is that if I stand no hope of getting job X, no matter how much I improve, I do really want to know that, so I can make choices about how much time to spend trying to get that job or jobs like it. It feels like a kindness to tell me I can do anything I put my mind to, but if it’s not true then you’re just setting me up for more pain in the future. (Similarly, saying “everyone should apply, even if you’re not sure you’re qualified” sounds like a kindness but does have a downside in terms of increasing the number of unsuccessful applicants; sometimes it’s worth it anyway, but the downside should be acknowledged.)
There is a sort of a trade-off to notifying people immediately or notifying them after a couple of days. My best guess is that it generally won’t make a difference for someone’s planning to be rejected from a job application in less than 24 hours or to be rejected within a few days. But there is probably a lot of variation in preferences from one person to another; maybe I am impacted by this more than average. I’m probably heavily influences by a typical mind fallacy here as well, as I am very sloppily generalizing from my own internal state.
I’ve had a few job applications that I submitted and then got rejected for an hour or two later, and emotionally that felt so much worse. But at the end of the day I think you are right that “your mileage may vary.”
This is a sloppy rough draft that I have had sitting in a Google doc for months, and I figured that if I don’t share it now, it will sit there forever. So please read this as a rough grouping of some brainstormy ideas, rather than as some sort of highly confident and well-polished thesis.
- - - - - -
What feedback do rejected applicants want?
From speaking with rejected job applicants within the EA ecosystem during the past year, I roughly conclude that they want feedback in two different ways:
The first way is just emotional care, which is really just a different way of saying “be kind rather than being mean or being neutral.”[1] They don’t want to feel bad, because rejection isn’t fun. Anybody who has been excluded from a group of friends, or kicked out of a company, or in any way excluded from something that you want to be included in knows that it can feel bad.[2] It feels even worse if you appear to meet the requirements of the job, put in time and effort to try really hard, care a lot about the community and the mission, perceive this as one of only a few paths available to you for more/higher impact, and then you get summarily excluded with a formulaic email template. There isn’t any feasible way to make a rejection feel great, but you can minimize how crappy it feels. Thank the candidates for their time/effort, and emphasize that you are rejecting this application for this role rather than rejecting this person in general. Don’t reject people immediately after their submission; wait a couple of days. If Alice submits a work trial task and less than 24 hours later you reject her, it feels to her like you barely glanced at her work, even if you spent several hours diligently going over it.
Improving. People want actionable feedback. If they lack a particular skill, they would like to know how to get better so that they can go learn that skill and then be a stronger candidate for this type of role in the future. If the main differentiator between candidates Alice and Bob is that Alice scored 50 points better on an IQ test or that Alice attended Impressive School while Bob attended No Name School, maybe don’t tell Bob that.[3] But if the main differentiator is that Alice has spent a year being a volunteer for the EA Virtual Program or that Alice is really good with spreadsheets or that Bob didn’t format his documents well, that is actionable, and gives the candidate a signal regarding how they can improve. Now the candidate knows something they can do to make become a more competitive candidate. They will practice their Excel skills and look up spreadsheet tutorials, they can get some volunteering experience with a relevant organization, and they look up how to use headers and to adjust line spacing. Think of this like a company investing in the local community college and sponsoring a professorship at the college: they are building a pipeline of potential future employees.
Here is a rough hierarchy of what, in an ideal world, I’d like to receive when I am rejected from a job application:
“Thanks for applying. We won’t be moving forward with your application. Although it is never fun to receive an email like this, we want to express appreciation for the time you spent on this selection process. Regarding why we choose to not move forward with your application, it looks like you don’t have as much experience directly related to X as the candidates we are moving forward with, and we also want someone who is able to Y. Getting experience with Y is challenging, but some ideas are here: [LINK].”
“Thanks for applying. We won’t be moving forward with your application. It looks like you don’t have as much experience directly related to X as the most competitive candidates, and we also want someone who is able to Y.”
“Thanks for applying. We won’t be moving forward with your application.”
That last bullet point is what most EA organizations send (according to conversations I’ve had with candidates, as well as my own experiences in EA hiring rounds). I have seen two or three that sometimes send rejections that are similar to the first or similar to the second.[4] If the first bullet point looks too challenging and you think that it would take too much staff time, then see if you can do the second bullet point: simply telling people why (although this will dependent on the context) can make rejections a lot less hurtful, and also points them in the right direction for how to get better.
I haven’t seen any EA orgs being mean in their rejections, but I have seen and heard of most of them being neutral.
I still remember how bad it felt being told that I couldn’t join a feminist reading group because they didn’t want any men there. I think that was totally understandable, but it still felt bad to be excluded. I remember not being able to join a professional networking group because I was older than the cutoff age (they required new members to be under 30, and I was 31 when I learned about it). These things happened years ago, and were not particularly influential in my life. But people remember being excluded.
Things that people cannot change with a reasonable amount of time and effort (or things that would require a time machine, such as what university someone attended) are generally not good pieces of feedback to give people. These things aren’t actionable.
Last I saw, the Centre for Effective Altruism and Animal Advocacy Careers both had systems in place helping them to do better than average. It has been a while since I’ve interaction with the internals of either of their hiring systems, but last I checked they both send useful and actionable feedback for at least some of their rejections.
I’m on board with a lot of your emotional care advice, but,,,
...I feel like your mileage may vary on this one. I don’t like being in suspense, and moreover it’s helpful from a planning perspective to know what’s up sooner rather than later. I’d say instead that if you want to signal that you spent time with someone’s application, do it by making sure your rejection is conspicuously specific (i.e. mentions features of the applicant or their submissions, even if only superficially).
I also think you missed an entire third category of reason to want feedback, which is that if I stand no hope of getting job X, no matter how much I improve, I do really want to know that, so I can make choices about how much time to spend trying to get that job or jobs like it. It feels like a kindness to tell me I can do anything I put my mind to, but if it’s not true then you’re just setting me up for more pain in the future. (Similarly, saying “everyone should apply, even if you’re not sure you’re qualified” sounds like a kindness but does have a downside in terms of increasing the number of unsuccessful applicants; sometimes it’s worth it anyway, but the downside should be acknowledged.)
There is a sort of a trade-off to notifying people immediately or notifying them after a couple of days. My best guess is that it generally won’t make a difference for someone’s planning to be rejected from a job application in less than 24 hours or to be rejected within a few days. But there is probably a lot of variation in preferences from one person to another; maybe I am impacted by this more than average. I’m probably heavily influences by a typical mind fallacy here as well, as I am very sloppily generalizing from my own internal state.
I’ve had a few job applications that I submitted and then got rejected for an hour or two later, and emotionally that felt so much worse. But at the end of the day I think you are right that “your mileage may vary.”
Good point! I hadn’t thought of that, but that would be very helpful feedback to have.