Error

Really excited to see this post come out! I think this is a really helpful guide to people who want to work on AI Alignment, and would have been pretty useful to me in the past.

Thank you for writing this! I am at the career stage where this guide is very useful (early undergrad), and I wish I had this one year ago when I started getting into EA.

Over all I think this is a good post. However this part surprised me.

However, I am personally worried about people skill-building for a couple of years and then not switching to doing the most valuable alignment work they can, because it can be easy to justify that your work is helping when it isn’t. This can happen even at labs that claim to have a safety focus! Working at any of Anthropic, DeepMind, Redwood Research, or OpenAI seems like a safe bet though.

I agree with the first bit. I’m also worried that people motivated to help with alignment end up contributing more to capability than alignment. But I’m very surprised by your list of what orgs to trust on this. Redwood Research seems fine, but the other ones?

Do you mean the specifically the safety teams of Anthropic, DeepMind and OpenAI or the full orgs? If you meant just the safety teams, I would appreciate if you made this explicit in the post, to prevent any misunderstanding. Especially since this seems to have become the go to post to give to people looking of AI Safety career guidance.

If you mean the full orgs, not just the safety team, I would like to know your reasoning.

Telling people to trust the three leading capabilities orgs seems very bad to me.

Amazing post! Thank you.

I just saw that 80000hours’ technical AI safety research career review was based on this post!

I have found this post super valuable when it came out, and I have sent it to lots of people who are starting out in technical AI safety since then.

Do you think now it makes more sense to send them the career review instead? Or does it still make sense for them to read both?

I know this post was written a while back, but I’m still finding it super helpful. thank you!

Thank you so much for taking your time to write this! As someone who’s seriously considering to leave their unfinished major in Chemistry behind to pursue AI alignment work, I can’t emphasize enough how much I appreciate this guide.

Thanks for highlighting text in bold!

Thanks for writing this. I think there’s a couple categories that I’d add to the types of technical work, which are both super important for this kind of technical research, and I think would make this kind of document much more broadly applicable.

I think the first is social scientists—in general more and more of technical research ends up intersecting some extremely difficult questions that have been well studied by social fields in the past. For example, working on Truthful/​Honest AI systems almost inevitably ends up in questions of epistemology or what constitutes acceptable evidence. Evaluations about whether an AI systems outputs or decisions are harmful can be difficult to evaluate, etc etc. I want to defer to https://​​distill.pub/​​2019/​​safety-needs-social-scientists/​​ for better arguments here, but mostly I want to clarify that technical AI alignment research needs social scientists.

The second is people who are experts in gathering and managing human data and human labelers. Data is one of the fuels of this kind of research, and human-provided data in particular is very valuable to technical alignment research. Human labelers can be used to evaluate a model (and decide whether or not it should be deployed) or identify issues in the outputs, etc. Hiring and training is difficult and complex. So I would include roles for folks that can help facilitate and accelerate human data processes as another important role in technical AI alignment research.