Thanks for all your hard work, Megan.
I’m reminded of this post from a few months ago: Does Sam make me want to renounce the actions of the EA community? No. Does your reaction? Absolutely.
And this point from a post Peter Wildeford wrote: “I think criticism of EA may be more discouraging than it is intended to be and we don’t think about this enough.”
In theory, the EA movement isn’t about us as EAs. It’s about doing good for others. But in practice, we’re all humans, and I think it’s human nature to have an expectation of recognition/gratitude when we’ve done an altruistic act. If instead of gratitude, we get a punishment in the form of a bad outcome or sharp words, that feels like a bait & switch.
My hypothesis is that being surrounded by other do-gooders makes the situation worse. You feel like you’re in a recognition deficit, many people around you feel the same way, and no one is injecting gratitude into the ecosystem to resolve the misery spiral. Internal debates exacerbate things, insofar as trying to understand someone else’s perspective depletes the same emotional resource that altruism does.
Anyway, most of that wasn’t very specific to your post—I’m just wondering if emphasizing “other-care” in addition to “self-care” would help us weather ups & downs.
And, thanks to all the EAs reading this for all the good you are doing.
On the topic of feedback… At Triplebyte, where I used to work as an interviewer, we would give feedback to every candidate who went through our technical phone screen. I wasn’t directly involved in this, but I can share my observations—I know some other EAs who worked at Triplebyte were more heavily involved, and maybe they can fill in details that I’m missing. My overall take is that offering feedback is a very good idea and EA orgs should at least experiment with it.
Offering feedback was a key selling point that allowed us to attract more applicants.
As an interviewer, I was supposed to be totally candid in my interview notes, and also completely avoid any feedback during the screening call itself. Someone else in the company (who wasn’t necessarily a programmer) would lightly edit those notes before emailing them—they wanted me to be 100% focused on making an accurate assessment, and leave the diplomacy to others. My takeaway is that giving feedback can likely be “outsourced”—you can have a contractor / ops person / comms person / intern / junior employee take notes on hiring discussions, then formulate diplomatic but accurate feedback for candidates.
My boss told me that the vast majority of candidates appreciated our feedback. I never heard of any candidate suing us, even though we were offering feedback on an industrial scale. I think occasionally candidates got upset, but they mostly insulated me from that unless they thought it would be valuable for me to hear—they wanted my notes to stay candid.
Jan writes: “when evaluating hundreds of applications, it is basically certain some errors are made, some credentials misunderstood, experiences not counted as they should, etc. - but even if the error rate is low, some people will rightfully complain, making hiring processes even more costly.” I think insofar as you have low confidence in your hiring pipeline, you should definitely be communicating this to candidates, so they don’t over-update on rejection. At Triplebyte, we had way more data to validate our process than I imagine any EA org has. But I believe that “our process is noisy and we know we’re rejecting good candidates” was part of the standard apologetic preamble to our feedback emails. (One of the worst parts of my job was constant anxiety that I was making the wrong call and unfairly harming a good candidate’s career.)
Relatedly… I’m in favor of orgs taking the time to give good feedback. It seems likely worthwhile as an investment in the human capital of the rejectee, the social capital of the community as a whole, and improved community retention. But I don’t think feedback needs to be good to be appreciated—especially if you make it clear if your feedback is low confidence. As a candidate, I’m often asking the question of which hoops I need to jump through in order to get a particular sort of job. If part of hoop-jumping means dealing with imperfect interviewers who aren’t getting an accurate impression of my skills, I want to know that so I can demonstrate my skills better.
But I also think that practices that help you give good feedback are quite similar to practices that make you a good interviewer in general. If your process doesn’t give candidates a solid chance to demonstrate their skills, that is something you should fix if you want to hire the best people! (And hearing from candidates whose skills were, in fact, judged inaccurately will help you fix it! BTW, I predict if you acknowledge your mistake and apologize, the candidate will get way less upset, even if you don’t end up hiring them.) A few more examples to demonstrate the point that interviewing and giving feedback are similar competencies:
Concrete examples are very useful for feedback. And I was trained to always have at least one concrete example to back up any given assessment, to avoid collecting fuzzy overall impressions that might be due to subconscious bias. (BTW, I only saw a candidate’s resume at the very end of the interview, which I think was helpful.)
Recording the interview (with the candidate’s consent), so you can review it as needed later, is another thing that helps with both objectives. (The vast majority of Triplebyte candidates were happy to have their interview recorded.)
Using objective, quantifiable metrics (or standard rubrics) makes your process better, and can also give candidates valuable info on their relative strengths and weaknesses. (Obviously you want to be diplomatic, e.g. if a candidate really struggled somewhere, I think we described their skills in that area as “developing” or something. We’d also give them links to resources to help them level up on that.)
At Triplebyte, we offered feedback to every candidate regardless of whether they asked for it. I once suggested to my boss that we should make it opt-in, because that would decrease the time cost on our side and also avoid offending candidates who didn’t actually want feedback. IIRC my boss didn’t really object to that thought. It wasn’t deemed a high-priority change, but I would suggest organizations creating a process from scratch make feedback opt-in.
BTW if any EA hiring managers have questions for me I’m happy to answer here, via direct message, or on a video call. I interviewed both generalist software engineers (tilted towards backend web development) and machine learning engineers.