Working in AI Safety fieldbuilding for 2 years. Mainly thinking about ops/generalist pipelines.
Mick
Karma: 102
Operations in AI Safety: A One-Year Perspective and Advice
What I’d want a fiscal sponsor to do well (also, some generic principles for ops)
What I’d want a fiscal sponsor to do well (also, some generic principles for ops)
EDIT: This is now a top-level post so that it’s easier to find and share.
Thank you for writing this! I had been missing a resource I could point people towards who were interested in Research Manager roles, and this post fills that gap perfectly.
If someone would read this and think to themselves “I might be a good RM”, what’s the most important thing you’d want them to check about themselves to see if they’re a good fit for the role?
My thoughts are similar to titotal’s above: I found it hard to get through. There are a lot of stock Claude/LLM phrases, such as the “It’s not this. It’s this” and the usage of “Reality check”, the use of slightly too uncommon synonyms, and the slightly too fancy vocabulary.
I think there’s value in LLM feedback but when it rewrites whole sections it usually starts to feel annoying to me. I don’t know if you have a “system prompt” for your Claude, but prompting it to preserve your voice much more, or just give you a specific list of improvements to implement might work. It could also be worth giving Claude some other things you’ve written as context for “your voice” and to give it strict instructions to avoid certain ways of writing.Some of the things I did like from the Claude version because they made it more skimmable and easier to figure out what was happening:
The weeks in the section headers
Key points bolded
The section recapping what you learned about career transitions
Relatedly, I think having a TL; DR at the top of posts is generally helpful
I struggle with the same perfectionism, but reading your original post, it does not seem net-negative to me. It works very well for the personal reflection blog post format, and is much more enjoyable to read. If you were applying for writing/blogging positions it would probably be too unpolished, but even then they wouldn’t care if you had older material that was less polished. If you’re concerned about it you could probably mostly mitigate it by adding a disclaimer at the top that you wrote it in a limited amount of time.
You also can’t really make a mistake in this kind of post because it is a personal reflection. It’s about your experience, rather than e.g. you presenting research results or carefully arguing for an opinion which would be much higher stakes and would require more carefulness. You can’t get your own experience wrong.
I think this post is very valuable as a resource for other people considering going to a future iteration of ARBOx or self-studying the ARENA curriculum. It reminds me a bit of the ML4Good experience reports [1] [2] [3] [4]
Thank you for writing this! I love how you’re sharing a lot of the unspoken parts about what starting an org is like, and the lessons are things anyone founding in AI Safety should be aware of before going down the path.
I’ve “only” been to 2 EAGs and 4 EAGx’s so take this with that as context
For previous EAGs I always booked my schedule full of 1-1′s to ask people about their experience, resolve uncertainties, and just generally network with people in similar roles. This EAG (NYC 2025) I didn’t find as many people on Swapcard that I wanted to talk to and received much less requests for 1-1s, so I also ended up having just 7 1-1s in total. This was a fun experiment. I found it much more relaxed, and I enjoyed being able to have spontaneous conversations with people I ran into, but I think overall I got less value out of this EAG than if I had booked more meetings: I have less actionable insights and met less people than during other EAG(x) conferences I have attended. However, I’m definitely in favour of less 1-1 cramming.
I do think if this was one of my first EAGs and I didn’t know anyone, I would’ve been quite lost without the structure of the 1-1′s and the explicit encouragement that it is normal to book a lot. I also feel weird about just joining a conversation in case it was people having a private 1-1. Having an improved spontaneous conversations area with bigger signs/cause area specific areas (or time slots?) sounds like a great solution for both of these problems.
Tangentially, my favourite meetups are also those where you just stand and mingle, ideally with specific areas in specific corners, rather than do forced speed meets or roundtable discussions. This makes it much easier to leave if you don’t like a conversation and move on to a different one until you find one you like.
I have an idea on a post with best practices for how student group leaders should interact with their Students’ Associations/Unions/Guilds/etc.
Before I started working on making AI go well, I worked in the Student Opportunities department of my Students’ Association and during this time, I interacted with a lot of student group leaders. However, I was never part of an EA Student group, so I don’t know what group leaders want to know, and I am not sure how well my knowledge transfers outside of my university.
If you’re a group leader and there’s anything you’re curious about, let me know, and worst-case I will send you a private answer; best case I’ll write a post.
For others: Theo and I got in contact and was added to the chat. Feel free to comment here if you’re also interested.
Having also done coaching, I do just frequently share the same links with most people I talk with. It does seem to help a little bit when you say to their face “this is a great resource” though, and even better if you can specifically justify to someone why this is a good resource for someone in their position.
I’m curious how well Stampy, 80ks AI advisor or Lens Academy’s coach do at this.
In my experience, you run into these problems when you try to do this:
not being specific enough
Being so specific the page becomes very long 3. Or, you have to put a lot of effort filtering for people what they should pay attention to (which depends on seniority, previous career, how bought into AI risk they are, what roles they’re interested in, restrictions in careers such as location, family, etc) that it becomes unwieldy
A lot of these resources being reading, which selects for certain kinds of people
Having to review a lot of different resources to find the best ones, among different kind of media .
The field moving so fast you have to keep this up to date, meaning you have to stay on top of current resources.
For reference, here’s the bare bones version we came up with for AI Safety Quest: https://aisafety.quest/#calls where we explicitly went for “short list of stable resources that are introductory, but require effort from the reader to go in depth”.
If you’re (re)starting a local EA group or running a local EA event, consider reaching out to people nearby according to the people directory or an EAG Swapcard.
There will be people who do not know about a local group or event and otherwise wouldn’t hear about it, and it’s pretty painless on both ends! Obviously don’t spam them, so (probably) only do this when (re)starting. I did this for an EA Edinburgh event and immediately got some responses.
To make this easier: Put your (approximate) location on the people directory so people can tell you about nearby events and you get them suggested by the forum, especially if you’re not in a hub!