Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Thanks for doing this coaching and sharing the update! Some answers from myself; I’m not sure how much my coworkers at CEA agree with me, so these are intended as a personal opinion rather than an official stance:
I spent a few minutes trying to think of engineers that I personally know for whom it’s so obvious they should earn to give that they shouldn’t even apply to EA organizations. I was only able to think of a handful, and they are all on the founding teams of $100M+ companies.
I advise myself (CTO/cofounder of a reasonably successful company) and engineers at top quant trading firms to strongly consider direct work. I can think of only one person who had an engineering offer from an EA organization I respected but I thought they should do earning to give instead.
A more common scenario is “earning to learn” – EA org’s don’t always have junior positions or provide mentorship, and many (probably most) junior EA engineers should be skilling up at for-profit organizations. Again here though, I think these people should at least try to apply for positions at EA organizations, and if they get rejected ask the hiring manager where they should skill up, then try to gain those skills elsewhere.
(I acknowledge that applying to a bunch of EA organizations and getting rejected from all of them really sucks; the above advice is what I think an impact-maximizing robot would do, but there might be psychological reasons why the people you are advising should not do it. To the extent that you can coach people to feel okay with rejection though, that seems helpful.)
For CEA:
The initial stage of the interview is leetcode-style algorithms questions. Typescript/JavaScript preferred, Python second choice. I haven’t given this a lot of thought, but the next time we hire for an engineer I might just outsource to Triplebyte. 1.1. I’m having difficulty calibrating exactly how hard our questions are. I think they are harder than “hard” leetcode questions. I can try to think of a better calibration point, if this would be useful to you.
The last stage is the person actually working with us for a day or two. We have an open source code base, so you can see verbatim what this looks like; e.g. this is what the most recent engineer we hired did during her interview.
Regarding whether people should work somewhere for some amount of time before applying to CEA: the email I send to people with limited experience is basically “to be respectful of your time, I’ll tell you that I think you probably won’t be a fit here, but it’s really hard to judge someone from their resume so if you want to try our trial task you should go for it.” Most people bail after that email without trying the task, and no one I’ve sent that email to has successfully completed it, but I’m sure there are people with zero work experience who would be capable of doing it.
Can...can you share one that you think is that hard?
+1 to this question.
Do you mean “hard” in a specific website?
Or perhaps you could make up an example question that is has a similar difficulty to whatever you ask?
Another example: I really like how Anthropic said “If you think you could write a substantial pull request for a major machine learning library, then major AI safety labs want to interview you today”
Leetcode has a “hard” tag on questions, I assume Ben meant these https://leetcode.com/problemset/all/?difficulty=HARD
I agree there is a huge variance, I have solved 19 last year and remember some being much much harder than others
Yes, I’m fairly sure this is what Ben meant. I’ve looked over some of the earlier versions of questions Ben was interested in, and I can confirm that the questions are harder than the average “Hard” questions on leetcode, which in turn is harder than typical interview questions at large tech companies. However, for people intimidated by this, I think there are ~3 things that claw back the difficulty somewhat:
The typical expected time to complete a BigTech interview question is about 30 minutes in a 45 minute interview (with ~15 minutes extra time to explain your solutions, ask questions of the interviewer, etc), iirc CEA gives you ~2-3h to solve their questions.
Most BigTech interviews are closed book on a whiteboard, iiuc CEA’s interviews are open book and you can use your preferred IDE.
(Only relevant for people familiar with leetcode questions). Leetcode questions have, for want of a better word, “idioms,” ie, common patterns that crop up fairly frequently across leetcode problems, including Leetcode Hard problems (e.g., sliding window, square root decomposition) but almost never real programming jobs. This makes Leetcode Hard problems easier for people who’ve already solved other LC problems, in ways that don’t generalize as well to solving algorithm puzzles that don’t come from the same generator.
One reason I asked about the difficulty of Ben’s problems, is that I want to calibrate myself to the EA tech talent pool when giving comments.
It would be good to know if many people here, with little practice, can solve unseen leetcode hard problems in a BigTech interview format (where you have to simultaneously create a verbal narrative throughout the interview).
If this is true, I will stop giving “advice” on any tech related post as this is out of my reference class.
Charles,
TL;DR: I would not stop giving tech advice just because you can’t solve hard leetcode puzzels
Longer:
If it influences your opinion:
I think I’m very good at leetcode puzzels
I give tech advice
I think those “skills” are almost totally unrelated.
It is very rare for me to actually teach someone how to solve leetcode
It is more common for me to just point out “cracking the coding interview” as a legit way to prepare for big-tech interviews, but knowing this is unrelated to my ability to solving the questions myself. (I know it because a Google recruiter recommended that book for me. I can elaborate)
Thank you for the thoughtful reply Yonatan!
I guess my intuition is that the correlation between being good at giving career advice for techies and being able to solve algorithms puzzles is only moderate, and very little of that is causal.
Fwiw I suspect this is pretty uncommon among EAs, including EAs in tech. I was unusually good at leetcode-style problems (both in general and especially relative to my ability to actually do useful work in programming), and I suspect my hit rate for novel LC Hard problems in 30 minutes was less than 50%*.
*I’ve regressed a lot in my algorithms puzzles ability, I’m not even confident I can consistently do LC Easys now.
As Linch says, we don’t use the interview format you describe. It’s all done asynchronously, and with less time pressure. I’m not well calibrated on how well the average EA does under the whiteboard format, but I would say that in our format:
~50% of applicants pass our 30 minute screen
5-10% pass our 2.5 hour in-depth task
~5% get an offer
Note that the people who apply for our jobs are probably not representative of who you would speak to on the Forum, so I’m not sure how helpful these statistics actually are.
Lorenzo was correct, I was referring to the classification of “hard” on leetcode.
A rough equivalent to our trial task is: implement a piece table in 2.5 hours from a scaffold with some basic tests, using Wikipedia/the Internet but you aren’t allowed to verbatim copy code. It’s done as a take-home, so you don’t have to think out loud or write on a whiteboard or whatever.
I spent a few minutes trying to think of an analog to the Anthropic statement; I think if you are able to write a substantial pull request to a popular npm package (or similar web-based framework) I’m probably interested in talking. (Though I want to reemphasize that people should value my time at ~0, and therefore apply even if they think they are unlikely to get an offer.)
In response to your questions, speaking with some (limited, worked as an not-100%-software engineer in the SF bay area with lots of SW eng friends) experience:
Large or VC-rich companies have recruiters, they’re fine to use. Lots of good but smaller companies don’t have recruiters. You can apply through their website but get a referral if possible.
A referral? Yes, most companies have an internal referral system, and if you can get any kind of referral whatsoever, this helps you.
If you hate networking, Triplebyte (https://triplebyte.com/) is pretty great. Take a programming test, get matched with top companies.
Something potentially non-obvious: there is an indirect and socially-acceptable way to ask for an internal referral. It goes something like this:
If you know someone who currently works at a company: you don’t have to ask indirectly. You can just say, “hey, will you refer me for [job] at [your company]?” You can offer to give more context on why you’re interested or what relevant background you have, but you may not need to.
If you have a tenuous connection[1] to someone who works at a company: it’s acceptable to contact them (over email if you have it, otherwise over LinkedIn) saying something like, “hey, I’ve been looking at [job] at [your company] and would love to learn more about what’s it like to work there. Any chance you’d have have time for a coffee chat / video call to talk about it in the next week?”
If they agree, aim to have a pleasant conversation about your mutual (tech / business) interest. Be a friendly, curious person who enthusiastically wants to hear about their job and company culture. This is not an interview in either direction!
Towards the end of your conversation, assuming you’re still interested in the job, say something like, “alright, thanks so much, this was so helpful. I think I will apply!” (pause here to see if they offer to refer you) “No pressure, but I figured it doesn’t hurt to ask: does [your company] have an internal referral system? Any chance you’d be willing to refer me for [job]?”
If you have a no connections at a company: it’s acceptable to look through your social networks (e.g. LinkedIn, Facebook) to see if anyone at the company in one degree of separation from you, and ask for an intro through your connection. For example, you might be in touch with a former co-worker who was in the same grad school lab as someone now working at the company. It’s okay to ask that former coworker something like, “hey, I’m not sure how in touch you are with [person], but I’m thinking of applying to [their company] and would like to learn more, any chance you’d be able to intro me to them?”. Make it clear that they can say no! If they say yes, you will probably get an intro email to the person, and then this turns into (2).
It is the case in the USA that a good candidate usually meets more than half of the listed job requirements, but a candidate who meets all of them is probably overqualified.
One thing I’ve found annoying / weird is that you don’t necessarily hear back from recruiters or others if you don’t get a job, even after an interview. This varies among companies, but for many of them you just… eventually assume that you didn’t get the job.
You may have seen this already, but Haseeb Qureshi’s writing on US tech careers for earn-to-give is top-notch stuff: https://haseebq.com/tech-careers/. I’ve forwarded his advice on negotiating job offers to SO many people.
on the level of, like, “took a class with them 5 years ago” or “volunteered with them at an event once”. These are not people you’re really in touch with.
Thank you very much, this is very useful for me, specifically that you also added the culture/social norms which there’s no way I’d guess myself
I have a mild amount of insight into getting algo trading software engineering jobs: I had an internship at one of these companies this last summer and got a full time offer from from several. (However, I’m planning on working on alignment).
Maybe this is all totally obvious, but regardless, my short summary for one path into this industry is something like:
Go to university in the US, ideally somewhere with prestige. Top public universities (e.g. university of Michigan) can be decent, but MIT or similar is better.
Try to build up an internship ladder to one of these places. Maybe go small company → small hedge fund or other finance → algo trading.
Once you get an internship, you should be basically good to go (assuming the company thought you did a good job). I think most people get return offers and once you have an internship at one of these places, it’s easy to get past the resume screen at other algo trading companies.
This means that internships after freshman/sophomore year can be really important. Of course, ideally, you would just intern at an algo trading place from the start, but these internships can be hard to get without the right prior internships.
Obvious advice is under rated, thank you for sharing this
Thoughts on data science questions from myself (worked 2 junior data science positions after ~1 year of self-study) and my circles (mostly DS/ML at tech orgs):
Data science work involves more uncertainty and longer iteration times (i.e. debugging workflow is more of an art and takes half a day as opposed to a minute). Does this sound like a pro or con?
DS/ML hiring managers care more about pedigree. Your resume may be auto-filtered by HR/scanning software if you don’t have a graduate degree in ML/math/stats/computational science. Do you have one/would this be of interest to you?
What areas of DS/ML are you most interested in? Depending on what that is, SWE and DS work may not be mutually exclusive: Look into positions in analytics engineering, ML engineering, data engineering, software engineering for telemetry/observability (common in organizations with more mature analytics departments).
Potentially useful blog post from a friend of mine, that mirrors the path I took: https://nicholasvadivelu.com/2020/08/30/learn-data-sci/
I’d be willing to mentor data scientists, though would probably only be useful to people more junior (i.e. finding their first job in data science).