Iâm sorry to hear about your negative experience with GiveWellâs hiring cycle.
I think that itâs easy to under-estimate how hard it is to hire well though. For comparison, you can honestly give all the same complaints about the hiring practice of my parent company (Google).
It is slow, with many friends of mine experiencing up to a year of gap between application and eventual decision.
Later interviewers have no context on your performance on earlier parts of the application. This is actually deliberate though, since we want to get independent signal at each step. I wouldnât be surprised if it was deliberate at GiveWell as well.
You often arenât told what is important at each interview stage. Youâre just posed technical or behavioral questions, and then you have to figure out whatâs important to solve the problem. Again, this is somewhat deliberate to see if candidates are able to think through what the important parts of an issue are.
You certainly arenât given feedback for improvement after rejection. An explicit part of interviewer training is noting that we shouldnât say anything about a candidateâs performance (good or bad) to the candidate, for fear of legal repercussions. Some EA orgs have chosen to give rejection feedback despite this, but it seems to be both not standard and not necessarily wise for the organization.
Interviewing and hiring just kind of sucks. Iâd love it if GiveWell was unusually excellent here, but I think that itâs at least important to recognize that their hiring practices are pretty normal.
An explicit part of interviewer training is noting that we shouldnât say anything about a candidateâs performance (good or bad) to the candidate, for fear of legal repercussions.
Legal repercussions for interview feedback have been discussed on the EA Forum in the past, e.g. in the comments of this post. The consensus seems to be that itâs not an issue either in theory or in practice. Certainly if your feedback is composed of lawyer-approved boilerplate, and only offered to candidates who ask for it, I think your legal risk is essentially nil. [Edit: Liability insurance could be used to mitigate downside risk even further.]
Recent relevant post: Keep EA high-trust. Something Iâve observed is that the trust relationship in EA seems very asymmetrical. Junior EAs are asked to put lots of trust in senior EAs, but senior EAs put very little trust in junior EAs. Not giving feedback because of very hypothetical lawsuit risk is a good example of that.
I think trust is a two-way street. So if we want EA to be high-trust, then senior EAs should be a bit more willing to trust junior EAs, including by giving interview feedback.
Recent relevant post: Keep EA high-trust. Something Iâve observed is that the trust relationship in EA seems very asymmetrical. Junior EAs are asked to put lots of trust in senior EAs, but senior EAs put very little trust in junior EAs. Not giving feedback because of very hypothetical lawsuit risk is a good example of that.
I am telling you what Google told me (and continues to tell new interviewers) as part of its interview training. You may believe that you know the law better than Google, but I am too risk averse to believe that I know the law better than them.
The legal risks consist almost entirely of situations where there is reasonable cause to suspect that the applicant has been discriminated due to some protected characteristic. In these situations the hiring party is incentivized to maximally control information in order to minimize potential evidence. Feedback could act as legal ammunition for the benefit of the discriminated candidate.
Because hiring organisations gain very little from giving feedback and instead lose time, effort, and assume more risk when doing it; itâs very common to forbid recruiters and interviewers from giving feedback entirely. Exaggerating the legal risks provides an effective explanation for doing this. The rule is typically absolute because otherwise recruiters may be tempted to give feedback out of niceness or a desire to help rejected candidates.
Also, Googleâs interpretation of the law is almost certainly made from Googleâs perspective and for Googleâs benefit â not from the perspective of what is the desired outcome of the law; or even more importantly, what is the underlying issue and how should we be trying to solve it to make the world better.
Google is generally quite risk-averse. My guess is that they donât give feedback because that is the norm for American companies, and because there is no upside for them. Iâd be surprised if their lawyers put more than 10 hours of legal research into this.
Another thought: Even if Googleâs lawyers did some research and said âyeah we could probably give feedbackâ, my model of Google is they would not start giving feedback.
Separately regarding trust, I donât feel obligated to trust senior EAs. I sometimes read the analyses of senior EAs and like them, so I start to trust them more. Trust based on seniority alone seems bad, could you give some examples where you feel senior EAs are asking folks to trust them without evidence?
How about a post like this one? Itâs not an analysis. Itâs an announcement from CEA that says theyâre reducing transparency around event admissions.
There may be evidence that CEA is generally trustworthy, but the post doesnât give any evidence that theyâre doing a good job with event admissions in particular. [In fact, it updates me in the opposite direction. My understanding of psychology research (e.g. as summarized by Daniel Kahneman in Thinking Fast and Slow) is that a decision like EAG admission will be made most effectively using some sort of statistical prediction rule. CEA doesnât give any indication that it is even collecting a dataset with which to find such a rule. The author of the post essentially states that CEA is still in the dark ages of making a purely subjective judgement.]
I guess I read that as a description of what theyâre doing rather asking me to trust them. CEA can choose the admission criteria they want, and after attending my first EAG earlier this year I felt like whatever criteria they were using seemed to broadly make for a valuable event for me as an attendee.
I think youâre really underestimating how hard giving useful feedback at scale is and how fraught it is. I would be more sympathetic if you were running or deeply involved with an organization that was doing better on this front. If you are, congrats and I am appreciative!
I guess I read that as a description of what theyâre doing rather asking me to trust them.
Itâs a description of how theyâre going to be less transparent. I think thatâs about as good as we can get, because if they hadnât described how they were going to be less transparent, there would be no post to share! All Iâd be able to say is âthey have a secretive vibeâ or something like that, which seems unsatisfactory.
(I do think the âsecretive vibeâ thing is true thoughâif we stop looking at posts that are published, and start looking at posts that arenât published, Iâd say the ratio of forum posts written by leadership in the past year to the number of EAs in leadership roles is quite low. Holden Karnofsky would be a notable exception here.)
So, Iâm not sure what would qualify as an answer to your âexamples where you feel senior EAs are asking folks to trust them without evidenceâ query at this point? You donât seem to think either the âKeep EA high-trustâ post or the âHow CEA approaches applications to our programsâ post qualifies.
I felt like whatever criteria they were using seemed to broadly make for a valuable event for me as an attendee.
Sounds like you have private info that theyâre trustworthy in this case. Thatâs great, but the post still represents senior EAs asking people to trust them without evidence.
Itâs not necessarily bad for senior EAs to be trusted, but I do think thereâs a severe trust imbalance and itâs causing significant problems.
I think youâre really underestimating how hard giving useful feedback at scale is
Can you explain why you think itâs hard? I am very familiar with a particular organization that does feedback at scale, like the UK Civil Serviceâthatâs the basis for my claims in this thread. I think maybe American organizations just arenât in the habit of giving feedback, and assume itâs much more difficult/âfraught than it actually is.
I think CEAâs mistake was âgetting into the weedsâ. Simply copy/âpasting boilerplate relevant to a particular application is a massive improvement compared to the baseline of no feedback. Categorize a random sample of rejected applications and for each category, identify an âEA virtueâ those rejects failed to demonstrate. Compose polite lawyer-approved boilerplate for each virtue. Then for a given reject who wants feedback, copy/âpaste the boilerplate for the virtues that werenât demonstrated sufficiently. Make it clear that you canât respond to follow-up emails. This would be pretty easy and have very minimal legal risk.
Alternatively, if CEA wants to get into the weeds in a fair way, then if an applicant asks for more info, flip 3 coins and if at least one lands tails, say no. If all coins are heads, get into the weeds publicly (anonymizing applicant identity by default) so people get a sense for whatâs going on. This could provide in-depth transparency without causing CEA to get overwhelmed.
A hypothetical example that I would view as asking for trust would be someone telling me not to join an organization, but not telling me why. Or claiming that another person shouldnât be trusted, without giving details. I personally very rarely see folks do this. An organization doing something different and explaining their reasoning (ex. giving feedback was not viewed as not a good ROI) is not asking for trust.
Regarding why giving feedback at scale is hard, most of these positions have at best vague evaluation metrics which usually bottom out in âhelp the organization achieve its goals.â Any specific criteria is very prone to being Goodharted. And the people who most need feedback are in my experience disproportionately likely to argue with you about it and make a stink to management. No need to trust me on this, just try out giving feedback at scale and see if itâs hard.
My admittedly limited understanding of the UK Civil Service suggests that itâs more amenable to quantization compared to GiveWell research analysts and Google software engineers. For example, if your job is working at the UK equivalent of a DMV, we could grade you based on number of customers served and a notion of error rate. That would seem pretty fair and somewhat hard to game. For a programmer, we could grade you based on tickets closed and bugs introduced. In contrast, this is absolute trash as the sole metric (although it does have some useful info).
Any specific criteria is very prone to being Goodharted.
I donât think CEA should share specific criteria. I think they should give rejects brief, tentative suggestions of how to develop as an EA in ways that will strengthen their application next time. Growth mindset over fixed mindset. Even a completely generic âmaybe you should get 80K advisingâ message for every reject would go a long way.
Earlier in this thread, I claimed that senior EAs put very little trust in junior EAs. The Goodharting discussion illustrates that well. The assumption is that if feedback is given, junior EAs will cynically game the system instead of using the feedback to grow in good faith. Iâm sure a few junior EAs will cynically game the system, but if the âcynical system-gamingâ people outweigh the âgood faith career growthâ people, we have much bigger problems than feedback. (And such an imbalance seems implausible in a movement focused on altruism.)
Iâd argue that lack of feedback actually invites cynical system-gaming, because youâre not giving people anywhere productive to direct their energies. And operating in a low-trust regime invites cynicism in general.
And the people who most need feedback are in my experience disproportionately likely to argue with you about it and make a stink to management.
Make it clear you wonât go back and forth this way.
This post explains why giving feedback is so important. If 5 minutes of feedback makes the difference for a reject getting bummed out and leaving the EA movement, it could be well worthwhile. My intuition is that this happens quite a bit, and CEA just isnât tracking it.
Re: making a stinkâthe person whoâs made the biggest stink in EA history is probably Ămile P. Torres. If you read the linked post, he seems to be in a cycle of: getting rejected, developing mental health issues from that, misbehaving due to mental health issues, then experiencing further rejections. (Again I refer you to the âCost of Rejectionâ postâmental health issues from rejection seem common, and lack of feedback is a big factor. As you mightâve guessed by this point, I was rejected for some EA stuff, and the mental health impact was much larger and longer than I wouldâve predicted in advance.)
I think we would prefer that rejects make a stink to management vs making a stink on social media. And 5 minutes of feedback to prevent someone from entering the same cycle Torres is in seems well worthwhile.
No need to trust me on this, just try out giving feedback at scale and see if itâs hard.
Again, I do have significant knowledge related to giving feedback at scale. It isnât nearly as hard as people say if you do it the right way.
My admittedly limited understanding of the UK Civil Service suggests that itâs more amenable to quantization compared to GiveWell research analysts and Google software engineers. For example, if your job is working at the UK equivalent of a DMV, we could grade you based on number of customers served and a notion of error rate. That would seem pretty fair and somewhat hard to game. For a programmer, we could grade you based on tickets closed and bugs introduced. In contrast, this is absolute trash as the sole metric (although it does have some useful info).
This seems like a red herring? I assume anyone applying for an analyst position at Givewell would be applying for a similar type of position at the Civil Service. White collar work may be hard to quantize, but that doesnât mean job performance canât be evaluated. And I donât see what evaluation of on-the-job performance has to do with our discussion.
I assume anyone applying for an analyst position at Givewell would be applying for a similar type of position at the Civil Service.
My experience with government positions is that they are legally required to have relatively formulaic hiring criteria. A benefit of this is that itâs easy to give feedback: you just screenshot your rubric and say âhere are the columns where you didnât get enough pointsâ.
So my guess is that even if there was literally the same position at GiveWell and the UK Civil Service it would be substantially easier to give feedback for the civil service one (which of course doesnât necessarily mean that GW shouldnât give feedback, just that they are meaningfully different reference classes).
I donât think CEA should share specific criteria. I think they should give rejects brief, tentative suggestions of how to develop as an EA in ways that will strengthen their application next time. Growth mindset over fixed mindset. Even a completely generic âmaybe you should get 80K advisingâ message for every reject would go a long way.
When we have a specific idea about what would improve someoneâs chances (like âyou didnât give much detail on your application, could you add more information?â) weâll often give it.
I guess you would rather they say âalwaysâ instead of âoften,â but otherwise it seems like what you want? And my recollection is that even the generic rejection emails do contain generic advice like linking to 80 K?
I guess this is kind of a tangent on the thread, but for what itâs worth Iâm not sure that EAG is actually doing something different than what you are suggesting.
(Note: I work for CEA, but not on the events team.)
This thread is the kind of tiring back and forth Iâm talking about. Please, try organizing feedback for 5k+ rejected applicants for something every year and then come back to tell me why Iâm wrong and it really is easy. I promise to humbly eat crow at that time.
For what itâs worth, Iâm also feeling quite frustrated. Iâve been repeatedly giving you details of how an organization Iâm very familiar with (canât say more without compromising anonymity) did exactly what you claim is so difficult, and nothing seems to get through.
I wonât trouble you with further replies in this thread :-)
You can see how that the lack of details is basically asking me to⊠trust you without evidence?
Edit: to use less âgotcha phrasing, anonymously claiming that another organization is doing better on feedback, but not telling me how, is asking for me to blindly trust you for very little reason.
I donât think feedback practices are widely considered secrets that have to be protected, and if your familiarity is with the UK Civil Service, thatâs a massive organization where you can easily give a description without unduly narrowing yourself down.
I can confirm that my experiences at Google is similar, as someone who both went through the application process as an applicant and was an interviewer (however, I was never on a hiring committee or explicitly responsible for hiring decisions). Including the parts about slowness, about intentionally not knowing how the candidate did in earlier stages (I believe weâre technically barred from reading earlier evaluations before submitting our own), and the part about being trained to be very strongly forbidden from giving candidate feedback.
Another thing Iâll add:
However, Givewellâs application process assumes that people work entirely in isolation.
[...]
There is no instruction to e.g. cover issue x, or focus on area y, despite the fact that in a real workplace those instructions would be provided (or at least discussed before the exercise).
Hence, despite Givewellâs claims that the exercises reflect work tasks, that clearly is not the case as they do not reflect how work is actually conducted in a well-functioning organisation. (And if it reflects how work is conducted in Givewell, then that just proves that Givewell is not a well-functioning organisation.)
I wonder if this is just a workplace cultural difference. In almost every job Iâve had, being able to independently come up with an adequate solution for tightly scoped problems given minimum or no additional task-specific instructions is sort of the baseline expectation of junior workers in the âcore rolesâ of the organizations (e.g. software engineers at a tech company, or researchers in an EA research org). Now itâs often better if thereâs more communications and people know or are instructed to seek help when theyâre confused, but neither software engineering nor research are inherently very minute-to-minute collaborative activities.
I personally agree with the assessment with the OP that EA orgs should give feedback for final-round applicants, and have pushed for it before. However, I donât think Googleâs hiring process is particularly dysfunctional, other than maybe the slowness (I do think Google is dysfunctional in a number of other ways, just not this one).
It makes no sense to compare Givewell and Google. Alphabet has around 130,000 employees. Givewell has what, a few dozen? Obviously organizational dysfunction grows with size. Givewell should be compared to small firms and other small non-profits. Now everyone recognizes that even big companies probably could be much more efficient and Google in particular has got extremely fat and lazy off the back of the massive profitability of searchâhence why it keeps buying interesting things only to fail to do anything with them and then kill them, so that should also be factored in.
Google, quite frankly, are also in a position to set terms in the hiring marketplace. They can fuck people around and frankly will always have an endless stream of quality talent wanting to go work there anyway. Givewell, by contrast, is going to be reliant on its own reputation and that of the wider of the EA movement to attract the people it wants. It is not in the same position and reputation matters.
I spent quite a bit of time last year looking into the hiring processes/ârecommendations of various big companies, and Google was generally the one I came away most impressed/âconvinced by, in terms of how much I expect their hiring decisions to correlate with employee quality. Iâd actually claim itâs much better than most other companies (of any size), and probably has had a positive influence on hiring as a whole through people copying their processes.
Iâm not convinced it needs to take as long as it does, and that might be evidence of bloat/â dysfunction. But other than that, I donât think Googleâs hiring process is dysfunctional.
People and organizations figure out things much harder than hiring well. Compare running one of the most used search engines on the planet with laying out a couple of assessments, a couple of people looking at them, a bit of communication by email and phone, and all run with reasonable promptness, much less than what one would expect from everyday postal services.
Iâm sorry to hear about your negative experience with GiveWellâs hiring cycle.
I think that itâs easy to under-estimate how hard it is to hire well though. For comparison, you can honestly give all the same complaints about the hiring practice of my parent company (Google).
It is slow, with many friends of mine experiencing up to a year of gap between application and eventual decision.
Later interviewers have no context on your performance on earlier parts of the application. This is actually deliberate though, since we want to get independent signal at each step. I wouldnât be surprised if it was deliberate at GiveWell as well.
You often arenât told what is important at each interview stage. Youâre just posed technical or behavioral questions, and then you have to figure out whatâs important to solve the problem. Again, this is somewhat deliberate to see if candidates are able to think through what the important parts of an issue are.
You certainly arenât given feedback for improvement after rejection. An explicit part of interviewer training is noting that we shouldnât say anything about a candidateâs performance (good or bad) to the candidate, for fear of legal repercussions. Some EA orgs have chosen to give rejection feedback despite this, but it seems to be both not standard and not necessarily wise for the organization.
Interviewing and hiring just kind of sucks. Iâd love it if GiveWell was unusually excellent here, but I think that itâs at least important to recognize that their hiring practices are pretty normal.
Legal repercussions for interview feedback have been discussed on the EA Forum in the past, e.g. in the comments of this post. The consensus seems to be that itâs not an issue either in theory or in practice. Certainly if your feedback is composed of lawyer-approved boilerplate, and only offered to candidates who ask for it, I think your legal risk is essentially nil. [Edit: Liability insurance could be used to mitigate downside risk even further.]
Recent relevant post: Keep EA high-trust. Something Iâve observed is that the trust relationship in EA seems very asymmetrical. Junior EAs are asked to put lots of trust in senior EAs, but senior EAs put very little trust in junior EAs. Not giving feedback because of very hypothetical lawsuit risk is a good example of that.
I think trust is a two-way street. So if we want EA to be high-trust, then senior EAs should be a bit more willing to trust junior EAs, including by giving interview feedback.
This is a good point, thanks for writing it
I am telling you what Google told me (and continues to tell new interviewers) as part of its interview training. You may believe that you know the law better than Google, but I am too risk averse to believe that I know the law better than them.
The legal risks consist almost entirely of situations where there is reasonable cause to suspect that the applicant has been discriminated due to some protected characteristic. In these situations the hiring party is incentivized to maximally control information in order to minimize potential evidence. Feedback could act as legal ammunition for the benefit of the discriminated candidate.
Because hiring organisations gain very little from giving feedback and instead lose time, effort, and assume more risk when doing it; itâs very common to forbid recruiters and interviewers from giving feedback entirely. Exaggerating the legal risks provides an effective explanation for doing this. The rule is typically absolute because otherwise recruiters may be tempted to give feedback out of niceness or a desire to help rejected candidates.
Also, Googleâs interpretation of the law is almost certainly made from Googleâs perspective and for Googleâs benefit â not from the perspective of what is the desired outcome of the law; or even more importantly, what is the underlying issue and how should we be trying to solve it to make the world better.
Google is generally quite risk-averse. My guess is that they donât give feedback because that is the norm for American companies, and because there is no upside for them. Iâd be surprised if their lawyers put more than 10 hours of legal research into this.
Another thought: Even if Googleâs lawyers did some research and said âyeah we could probably give feedbackâ, my model of Google is they would not start giving feedback.
Separately regarding trust, I donât feel obligated to trust senior EAs. I sometimes read the analyses of senior EAs and like them, so I start to trust them more. Trust based on seniority alone seems bad, could you give some examples where you feel senior EAs are asking folks to trust them without evidence?
How about a post like this one? Itâs not an analysis. Itâs an announcement from CEA that says theyâre reducing transparency around event admissions.
There may be evidence that CEA is generally trustworthy, but the post doesnât give any evidence that theyâre doing a good job with event admissions in particular. [In fact, it updates me in the opposite direction. My understanding of psychology research (e.g. as summarized by Daniel Kahneman in Thinking Fast and Slow) is that a decision like EAG admission will be made most effectively using some sort of statistical prediction rule. CEA doesnât give any indication that it is even collecting a dataset with which to find such a rule. The author of the post essentially states that CEA is still in the dark ages of making a purely subjective judgement.]
I guess I read that as a description of what theyâre doing rather asking me to trust them. CEA can choose the admission criteria they want, and after attending my first EAG earlier this year I felt like whatever criteria they were using seemed to broadly make for a valuable event for me as an attendee.
I think youâre really underestimating how hard giving useful feedback at scale is and how fraught it is. I would be more sympathetic if you were running or deeply involved with an organization that was doing better on this front. If you are, congrats and I am appreciative!
Itâs a description of how theyâre going to be less transparent. I think thatâs about as good as we can get, because if they hadnât described how they were going to be less transparent, there would be no post to share! All Iâd be able to say is âthey have a secretive vibeâ or something like that, which seems unsatisfactory.
(I do think the âsecretive vibeâ thing is true thoughâif we stop looking at posts that are published, and start looking at posts that arenât published, Iâd say the ratio of forum posts written by leadership in the past year to the number of EAs in leadership roles is quite low. Holden Karnofsky would be a notable exception here.)
So, Iâm not sure what would qualify as an answer to your âexamples where you feel senior EAs are asking folks to trust them without evidenceâ query at this point? You donât seem to think either the âKeep EA high-trustâ post or the âHow CEA approaches applications to our programsâ post qualifies.
Sounds like you have private info that theyâre trustworthy in this case. Thatâs great, but the post still represents senior EAs asking people to trust them without evidence.
Itâs not necessarily bad for senior EAs to be trusted, but I do think thereâs a severe trust imbalance and itâs causing significant problems.
Can you explain why you think itâs hard? I am very familiar with a particular organization that does feedback at scale, like the UK Civil Serviceâthatâs the basis for my claims in this thread. I think maybe American organizations just arenât in the habit of giving feedback, and assume itâs much more difficult/âfraught than it actually is.
I think CEAâs mistake was âgetting into the weedsâ. Simply copy/âpasting boilerplate relevant to a particular application is a massive improvement compared to the baseline of no feedback. Categorize a random sample of rejected applications and for each category, identify an âEA virtueâ those rejects failed to demonstrate. Compose polite lawyer-approved boilerplate for each virtue. Then for a given reject who wants feedback, copy/âpaste the boilerplate for the virtues that werenât demonstrated sufficiently. Make it clear that you canât respond to follow-up emails. This would be pretty easy and have very minimal legal risk.
Alternatively, if CEA wants to get into the weeds in a fair way, then if an applicant asks for more info, flip 3 coins and if at least one lands tails, say no. If all coins are heads, get into the weeds publicly (anonymizing applicant identity by default) so people get a sense for whatâs going on. This could provide in-depth transparency without causing CEA to get overwhelmed.
A hypothetical example that I would view as asking for trust would be someone telling me not to join an organization, but not telling me why. Or claiming that another person shouldnât be trusted, without giving details. I personally very rarely see folks do this. An organization doing something different and explaining their reasoning (ex. giving feedback was not viewed as not a good ROI) is not asking for trust.
Regarding why giving feedback at scale is hard, most of these positions have at best vague evaluation metrics which usually bottom out in âhelp the organization achieve its goals.â Any specific criteria is very prone to being Goodharted. And the people who most need feedback are in my experience disproportionately likely to argue with you about it and make a stink to management. No need to trust me on this, just try out giving feedback at scale and see if itâs hard.
My admittedly limited understanding of the UK Civil Service suggests that itâs more amenable to quantization compared to GiveWell research analysts and Google software engineers. For example, if your job is working at the UK equivalent of a DMV, we could grade you based on number of customers served and a notion of error rate. That would seem pretty fair and somewhat hard to game. For a programmer, we could grade you based on tickets closed and bugs introduced. In contrast, this is absolute trash as the sole metric (although it does have some useful info).
I donât think CEA should share specific criteria. I think they should give rejects brief, tentative suggestions of how to develop as an EA in ways that will strengthen their application next time. Growth mindset over fixed mindset. Even a completely generic âmaybe you should get 80K advisingâ message for every reject would go a long way.
Earlier in this thread, I claimed that senior EAs put very little trust in junior EAs. The Goodharting discussion illustrates that well. The assumption is that if feedback is given, junior EAs will cynically game the system instead of using the feedback to grow in good faith. Iâm sure a few junior EAs will cynically game the system, but if the âcynical system-gamingâ people outweigh the âgood faith career growthâ people, we have much bigger problems than feedback. (And such an imbalance seems implausible in a movement focused on altruism.)
Iâd argue that lack of feedback actually invites cynical system-gaming, because youâre not giving people anywhere productive to direct their energies. And operating in a low-trust regime invites cynicism in general.
Make it clear you wonât go back and forth this way.
This post explains why giving feedback is so important. If 5 minutes of feedback makes the difference for a reject getting bummed out and leaving the EA movement, it could be well worthwhile. My intuition is that this happens quite a bit, and CEA just isnât tracking it.
Re: making a stinkâthe person whoâs made the biggest stink in EA history is probably Ămile P. Torres. If you read the linked post, he seems to be in a cycle of: getting rejected, developing mental health issues from that, misbehaving due to mental health issues, then experiencing further rejections. (Again I refer you to the âCost of Rejectionâ postâmental health issues from rejection seem common, and lack of feedback is a big factor. As you mightâve guessed by this point, I was rejected for some EA stuff, and the mental health impact was much larger and longer than I wouldâve predicted in advance.)
I think we would prefer that rejects make a stink to management vs making a stink on social media. And 5 minutes of feedback to prevent someone from entering the same cycle Torres is in seems well worthwhile.
Again, I do have significant knowledge related to giving feedback at scale. It isnât nearly as hard as people say if you do it the right way.
This seems like a red herring? I assume anyone applying for an analyst position at Givewell would be applying for a similar type of position at the Civil Service. White collar work may be hard to quantize, but that doesnât mean job performance canât be evaluated. And I donât see what evaluation of on-the-job performance has to do with our discussion.
My experience with government positions is that they are legally required to have relatively formulaic hiring criteria. A benefit of this is that itâs easy to give feedback: you just screenshot your rubric and say âhere are the columns where you didnât get enough pointsâ.
So my guess is that even if there was literally the same position at GiveWell and the UK Civil Service it would be substantially easier to give feedback for the civil service one (which of course doesnât necessarily mean that GW shouldnât give feedback, just that they are meaningfully different reference classes).
The post you linked says:
I guess you would rather they say âalwaysâ instead of âoften,â but otherwise it seems like what you want? And my recollection is that even the generic rejection emails do contain generic advice like linking to 80 K?
I guess this is kind of a tangent on the thread, but for what itâs worth Iâm not sure that EAG is actually doing something different than what you are suggesting.
(Note: I work for CEA, but not on the events team.)
This thread is the kind of tiring back and forth Iâm talking about. Please, try organizing feedback for 5k+ rejected applicants for something every year and then come back to tell me why Iâm wrong and it really is easy. I promise to humbly eat crow at that time.
For what itâs worth, Iâm also feeling quite frustrated. Iâve been repeatedly giving you details of how an organization Iâm very familiar with (canât say more without compromising anonymity) did exactly what you claim is so difficult, and nothing seems to get through.
I wonât trouble you with further replies in this thread :-)
You can see how that the lack of details is basically asking me to⊠trust you without evidence?
Edit: to use less âgotcha phrasing, anonymously claiming that another organization is doing better on feedback, but not telling me how, is asking for me to blindly trust you for very little reason.
I donât think feedback practices are widely considered secrets that have to be protected, and if your familiarity is with the UK Civil Service, thatâs a massive organization where you can easily give a description without unduly narrowing yourself down.
I can confirm that my experiences at Google is similar, as someone who both went through the application process as an applicant and was an interviewer (however, I was never on a hiring committee or explicitly responsible for hiring decisions). Including the parts about slowness, about intentionally not knowing how the candidate did in earlier stages (I believe weâre technically barred from reading earlier evaluations before submitting our own), and the part about being trained to be very strongly forbidden from giving candidate feedback.
Another thing Iâll add:
I wonder if this is just a workplace cultural difference. In almost every job Iâve had, being able to independently come up with an adequate solution for tightly scoped problems given minimum or no additional task-specific instructions is sort of the baseline expectation of junior workers in the âcore rolesâ of the organizations (e.g. software engineers at a tech company, or researchers in an EA research org). Now itâs often better if thereâs more communications and people know or are instructed to seek help when theyâre confused, but neither software engineering nor research are inherently very minute-to-minute collaborative activities.
I personally agree with the assessment with the OP that EA orgs should give feedback for final-round applicants, and have pushed for it before. However, I donât think Googleâs hiring process is particularly dysfunctional, other than maybe the slowness (I do think Google is dysfunctional in a number of other ways, just not this one).
It makes no sense to compare Givewell and Google. Alphabet has around 130,000 employees. Givewell has what, a few dozen? Obviously organizational dysfunction grows with size. Givewell should be compared to small firms and other small non-profits. Now everyone recognizes that even big companies probably could be much more efficient and Google in particular has got extremely fat and lazy off the back of the massive profitability of searchâhence why it keeps buying interesting things only to fail to do anything with them and then kill them, so that should also be factored in.
Google, quite frankly, are also in a position to set terms in the hiring marketplace. They can fuck people around and frankly will always have an endless stream of quality talent wanting to go work there anyway. Givewell, by contrast, is going to be reliant on its own reputation and that of the wider of the EA movement to attract the people it wants. It is not in the same position and reputation matters.
I spent quite a bit of time last year looking into the hiring processes/ârecommendations of various big companies, and Google was generally the one I came away most impressed/âconvinced by, in terms of how much I expect their hiring decisions to correlate with employee quality. Iâd actually claim itâs much better than most other companies (of any size), and probably has had a positive influence on hiring as a whole through people copying their processes.
Iâm not convinced it needs to take as long as it does, and that might be evidence of bloat/â dysfunction. But other than that, I donât think Googleâs hiring process is dysfunctional.
People and organizations figure out things much harder than hiring well. Compare running one of the most used search engines on the planet with laying out a couple of assessments, a couple of people looking at them, a bit of communication by email and phone, and all run with reasonable promptness, much less than what one would expect from everyday postal services.