This text is the polished and organized version of me trying to figure out what is stopping me from thinking clearly about AI safety. If you are looking for interesting and novel AI safety arguments, stop here and go read something else. However, if you are curious on how a person can engage with AI safety arguments without forming a coherent opinion about it, then read on.
Second place: How EA at Georgia Tech got 100+ program participants in our second semester of existence by mic
Just counting this semester, I think about 13 out of 33 participants of AI Safety Fundamentals plan on working in AI safety, and at least 6 of our other participants plan on working in a longtermist priority area. I think about 8 out of 32 of our participants from our Fall 2021 intro program plan on pursuing a high-impact career. Overall, I’m feeling optimistic about how much new EA university groups can grow and about how many university students the community can get engaged with EA!
I also think that at major CS universities, we can get a good number of people participating in the AI Safety Fundamentals alignment program, and I’d be excited to see more AI safety club activities at other universities. (See more at AI safety field-building at universities.)
One year ago, we hypothesized that a name change could increase the success of EA university groups. Now, one year later, we have gathered data and want to share the results of our naming experiment with other community builders.
Let’s start with the overall conclusion: in contrast to our initial post, we now believe that the advantages of our name change did not outweigh the disadvantages. A survey of our alumni showed that the advantages were smaller than hypothesized. In addition, we stumbled on some unanticipated difficulties during the year. In this post, we will briefly refresh your memory on our initial line of thinking, followed by our experiences of the year. We hope this follow-up helps others to crystalize their thoughts about names for their groups or organizations.
I lean towards using Laplace’s rule of succession to start quantifying whether it’s the hinge of history.[1] That means since I’m focused on the hinge of the future, the initial odds this century is the most influential future century would be ½.
Will MacAskill thinks there’s a 0.1 to 1% chance the current century is the hinge of history. Toby Ord thinks there’s a ⅙ chance there’ll be an existential catastrophe by 2120.
Yet Ord’s biggest disagreement with MacAskill isn’t related to climate change, nuclear war, pandemics or AI. It’s about how to set a Bayesian prior.
University students on an academic calendar in the US often start applying to summer opportunities as early as July or August and have their summer plans completely settled by January or February. For example: prestigious finance, consulting, and software engineering internships typically take most of their applications prior to December (or even sooner!), and funded summer research opportunities such as REUs typically have application deadlines in early February.
By contrast, many high-quality opportunities for early-career EAs are on a slightly later timescale.
In order to make the most of the talent pool at top universities, EA summer internship programs might want to consider making a conscious effort to process applications on an earlier timescale. Ideally, they would do some sort of rolling admissions, starting in August-October, in order to accommodate for both the fall-winter and winter-spring application cycles.
Congratulations to the winners, and to everyone who participated! Also thanks for your patience—there were a few unexpected delays in announcing. Winners have received an email about payment, which will be distributed in the next 1-2 weeks.
A few miscellaneous points/reflections:
I’m very grateful to Aris Richardson and Miranda Zhang, who served as judges for the contest.
I was surprised by the number of submissions. (Before running the contest, I guessed that I’d receive 15-25).
I’m excited to see more contests that focus on producing object-level progress. (In particular, since developing the idea for this contest, my own cause prioritization has shifted heavily toward AI safety. It’ll be interesting to see if AI safety contests & prizes can be used to (a) produce tangible progress or (b) expose more people to the arguments around risks from advanced AI systems.
Winners of the community-building writing contest
The community-building writing contest received 53 submissions. Here are the winners:
First Place: How I failed to form views on AI safety by Ada-Maaria Hyvärinen
Second place: How EA at Georgia Tech got 100+ program participants in our second semester of existence by mic
Third place: “Should you use EA in your group name?” An update on PISE’s naming experiment by Amarins and Joris P
Fourth place: Grappling With The Hinge Of History Part 2: Bayesian Dilemma by Matt Goldwater
Fifth place: Announce summer EA internships farther in advance by electroswing
A few miscellaneous points/reflections:
I’m very grateful to Aris Richardson and Miranda Zhang, who served as judges for the contest.
I was surprised by the number of submissions. (Before running the contest, I guessed that I’d receive 15-25).
I’m excited to see more contests that focus on producing object-level progress. (In particular, since developing the idea for this contest, my own cause prioritization has shifted heavily toward AI safety. It’ll be interesting to see if AI safety contests & prizes can be used to (a) produce tangible progress or (b) expose more people to the arguments around risks from advanced AI systems.