Thanks a lot for writing this!
I read this post back in 2020 and I’ve been using FVP regularly since then, so you’ve had a decent bit of impact on my life!
Magdalena Wache
Karma: 271
Technical AI Safety Research Landscape [Slides]
As a temporary measure in response to uncertainty about our future funding levels, we’ve put the bottom ~40% of grants above our current funding bar on hold.
Did you notify these applicants about their applications being put on hold?
(I just talked to an LTFF applicant who didn’t know about this measure, and they suspected that their application might be one of those which have been put on hold.)
Update: We did a follow-up survey 4 months after the retreat, asking participants what impact the retreat had on them. Here is a summary of the responses:
36 people responded (~54% of participants)
Personal prioritization
7 people reported their prioritization which area to work in was influenced by the retreat
One person quit their previous job to do a career transition to AI safety
One person decided to keep working on their current topic rather than switch as they had considered before
One person started to work on a research topic they wouldn’t have focused on otherwise
Connections
26 people mentioned they made connections they found useful, e.g. for research discussions.
7 people started research collaborations with people from the retreat
Applying for training programs and funding
5 people applied to SERI MATS as a result of the retreat
4 people got accepted into the online training program
2 people got accepted into the in-person research program
4 people applied for funding (1 accepted, 1 rejected, 2 unclear)
One person became a teaching assistant for the ML safety course
Concrete projects
At least 2 people took the AGISF 201 course
4 people started field-building projects
3 people wrote a post, and one person created a youtube channel
One is unsure whether their post was a result of the retreat
For one it was their first ever post
Other/Miscellaneous
3 people reported the retreat didn’t change much for them personally, but they think they were able to help others
All three were mentioned by others in the feedback as being particularly helpful
13 people said they got a better picture of the AI safety landscape in general
6 people reported they got a motivation boost from feeling part of a community of like-minded people
3 people mentioned they went to EAG London because of the retreat
One person shifted focus within their area of work
One person got useful experimental results they think they wouldn’t have gotten without the input from the retreat
(Note that the survey consisted of a free text field, and the categories I made up for summarizing the results are pretty subjective. Also note that some people mentioned that they started projects or applied for things, but would have done so even without the retreat. I did not include those in my count.)
I think these results are really helpful to improve the picture of how such a retreat is valuable for people!
The biggest surprise for me is the amount of tangible results (research collaborations, getting into SERI MATS, concrete projects, career changes) that came out of the retreat. There were 9 people who said in the feedback form that they want to start a project as a result of the retreat (the form they filled out right at the end of the retreat). I would have guessed that maybe 5 would actually follow through after the initial motivation directly after the retreat has worn off. And I think that would have been a very good result already. Instead, it was more than 9 people who started projects after the retreat!
I still have the intuition that the more vague effects of the retreat (such as feeling part of a community) are really important, potentially more so than the tangible outcomes. And these vague things are probably not measured very well by a survey that asks “What impact did the retreat have on you?”, because it is just easier to remember concrete than vague things.
Overall, these survey results make me even more excited about this kind of AI safety retreat!
Hi JJ, just wanted to say a big thank you for all your work!
I used the fiscal sponsorship, the free health coaching, the AISS newsletter, and the alignment slack, and found all of them super helpful! And probably I’ve used even more things which of I’m not even aware that they are coming from AISS.
I particularly appreciate your lots of links page. It’s one of my favorite resources for an overview of which things exist in the AI safety ecosystem.
I hope whatever you are up to next goes well!
I think the value comes from:
connections made
participants made 6.7 new connections on average (Question in the feedback form was “How many additional people working on AI safety would you feel comfortable asking for a small professional favor after this retreat? E.g. having a call to discuss research ideas, giving feedback on a draft, or other tasks that take around 30 minutes...”).
18 people mentioned their plans to reach out to people from the retreat for feedback on posts or career plans, for research discussions or for collaborations.
motivation boost
projects/collaborations started
9 people mentioned projects (Writing posts, doing experiments, and field building projects) they would start as a result of the retreat.
participants getting a better map of the space
which people exist, what kinds of things they are good at, what they have worked on, what you can ask them about.
What projects and what research exist.
A general sense of “getting more pieces of the puzzle”.
I think this is hard to measure, but really valuable.
I find it hard to say which of these is most important, and they are also highly entangled with each other
I just saw that 80000hours’ technical AI safety research career review was based on this post!
I have found this post super valuable when it came out, and I have sent it to lots of people who are starting out in technical AI safety since then.
Do you think now it makes more sense to send them the career review instead? Or does it still make sense for them to read both?