(Posting in a personal capacity unless stated otherwise.) I help allocate Open Phil’s resources to improve the governance of AI with a focus on avoiding catastrophic outcomes. Formerly co-founder of the Cambridge Boston Alignment Initiative, which supports AI alignment/safety research and outreach programs at Harvard, MIT, and beyond, co-president of Harvard EA, Director of Governance Programs at the Harvard AI Safety Team and MIT AI Alignment, and occasional AI governance researcher. I’m also a proud GWWC pledger and vegan.
tlevin
Karma: 1,392
Since you published this on April 1, I read the headline, thought it was satire, and laughed out loud, but it turns out this is a great post! Big fan of time-time tradeoffs.
University Groups Should Do More Retreats
Re: “I’d also encourage the more “senior people” to join retreats from time to time,” absolutely; not just (or even primarily) because you can provide value, but because retreats continue to be very useful in sharpening your cause prioritization, increasing your EA context, and building high-trust relationships with other EAs well after you’re “senior”!
Yep, great questions—thanks, Michael. To respond to your first thing, I definitely don’t expect that they’ll have those effects on everybody, just that they are much more likely to do so than pretty much any other standard EA group programming.
Depends on the retreat. HEA’s spring retreat (50 registrations, ~32 attendees) involved booking and communicating with a retreat center (which took probably 3-4 hours), probably 5-6 hours of time communicating with attendees, and like 2 hours planning programming. I ran a policy retreat in DC that was much more time-consuming, probably like 35 hours in figuring out logistics, communicating with guests, etc. I would guess the latter would do better on CBA (unless policy turns out to be very low-value).
I think scenic walks are probably the closest thing you can do on campus, but you definitely don’t get 80% of the value (even on a per-organizer-time basis). You get to tailor the conversation to their exact interests, but it’s not really the kind of sustained interaction in a self-contained social world that retreats offer.
Not with much confidence. I get the sense that the median person gets slightly more into EA but I guess like 5-10% of attendees can have major priorities shifts on the level of “EA seems like a cool way of thinking about climate policy” to “holy shit, x-risk.” I personally have shifted in a couple ways after retreats — from “optimize my time in grad school for generic policy career provided that I make some attempt at EA community-building” to “EA community-building should be one of my top two priorities” after the group organizer retreat and from “probably will work in biosecurity” to “probably will work in AI policy or EA meta” after Icecone.
It was very much an 80-20′d thing due to organizer capacity. The schedule was something like:
Friday evening arrivals + informal hangouts + board games (e.g. Pandemic)
Saturday morning: opening session, hikes/informal hangouts
Saturday afternoon: three sessions, each with multiple options:
1-on-1 walks, Updating Session, AI policy workshop
1-on-1 walks, Concept Swap, forecasting workshop
1-on-1 walks, AI policy workshop
Saturday evening: Hamming Circles, informal hangouts feat. hot tub and fire pit
Sunday morning: walks/hangouts
Sunday afternoon: career reflection, closing session, departure
Great post, possibly essential reading for community-builders; adding a link to this in several of my drafts + my retreat post. I think another important thing for CBers is to create a culture where changing your mind is high-status and having strongly held opinions without good reasons is not, which is basically the opposite of the broader culture (though I think EA does a good job of this overall). Ways I’ve tried to do this in settings with EA newcomers:
1) excitedly changing your mind—thinking of a Robi Rahmanism “The last time I changed my mind about something was right now.” This doesn’t just model openness; it also makes changing your mind a two-way street, rather than you having all the answers and they just need to learn from you, which I think makes it less identity-threatening or embarrassing to change your mind.
2) saying, in conversations with already-bought-in EAs that are in front of newcomers, things like “Hmm, I think you’re under-updating.” This shows that we expect longtime EAs to keep evaluating new evidence (and that we are comfortable disagreeing with each other) rather than just to memorize a catechism.
Also should note that we had a bit of a head start: I had organized the DC retreat one month earlier so had some recent experience, we had lots of excited EAs already so we didn’t even try to get professional EAs and we decided casual hangouts were probably very high-value, and the organizing team basically had workshops ready to go. We also had it at a retreat center that provided food (though not snacks). If any of these were different it would have taken much longer to plan.
Thanks, added to resources!
Yes, true, avoiding jargon is important!
I’ve seen the time-money tradeoff reach some pretty extreme, scope-insensitive conclusions. People correctly recognize that it’s not worth 30 minutes of time at a multi-organizer meeting to try to shave $10 off a food order, but they extrapolate this to it not being worth a few hours of solo organizer time to save thousands of dollars. I think people should probably adopt some kind of heuristic about how many EA dollars their EA time is worth and stick to it, even when it produces the unpleasant/unflattering conclusion that you should spend time to save money.
Also want to highlight “For example, we should avoid the framing of ‘people with money want to pay for you to do X’ and replace this with an explanation of why X matters a lot and why we don’t want anyone to be deterred from doing X if the costs are prohibitive” as what I think is the most clearly correct and actionable suggestion here.
With the caveat that this is obviously flawed data because the sample is “people who came to an all-expenses-paid retreat,” I think it’s useful to provide some actual data Harvard EA collected at our spring retreat. I was slightly concerned that the spending would rub people the wrong way, so I included as one of our anonymous feedback questions, “How much did the spending of money at this retreat make you feel uncomfortable [on a scale of 1 to 10]?” All 18 survey answerers provided an answer. Mean: 3.1. Median: 3. Mode: 1. High: 9.
I think it’s also worth noting that in response to the first question, “What did you think of the retreat overall?”, nobody mentioned money, including the person who answered 9 (who said “Excellent arrangements, well thought out, meticulous planning”). On the question “Imagine you’re on the team planning the next retreat, and it’s the first meeting. Fill in the blank: “One thing I think we could improve from the last retreat is ____”,” nobody volunteered spending less money; several suggestions involved adding things that would cost more money, including the person who answered 9, who suggested adding daily rapid tests. The question “Did participating in this retreat make you feel more or less like you want to be part of the EA community?” received mean 8.3, median 9, including a 9 from the person who felt most uncomfortable about the spending.
I concluded from this survey that, again, with the caveats for selection bias, the spending was not alienating people at the retreat, and especially not alienating enough to significantly affect their engagement with EA.
So glad to hear my post helped convince you to do this and that it went well!
Hmm, this does seem possible and maybe more than 50% likely. Reasons to think it might not be the case is that I know this person was fairly new to EA, not a longtermist, and somebody asked a clarifying question about this question that I think I answered in a clarifying way, but may not have clarified the direction of the scale. I don’t know!
This comment co-written with Jake McKinnon:
The post seems obviously true when the lifeguards are the general experts and authorities, who just tend not to see or care about the drowning children at all. It’s more ambiguous when the lifeguards are highly-regarded EAs.
It’s super important to try to get EAs to be more agentic and skeptical that more established people “have things under control.” In my model, the median EA is probably too deferential and should be nudged in the direction of “go save the children even though the lifeguards are ignoring them.” People need to be building their own models (even if they start by copying someone else’s model, which is better than copying their outputs!) so they can identify the cases where the lifeguards are messing up.
However, sometimes the lifeguards aren’t saving the children because the water is full of alligators or something. Like, lots of the initial ideas that very early EAs have about how to save the child are in fact ignorant about the nature of the problem (a common one is a version of “let’s just build the aligned AI first”). If people overcorrect to “the lifeguards aren’t doing anything,” then when the lifeguards tell them why their idea is dangerous, they’ll ignore them.
The synthesis here is something like: it’s very important that you understand why the lifeguards aren’t saving the children. Sometimes it’s because they’re missing key information, not personally well-suited to the task, exhausted from saving other children, or making a prioritization/judgment error in a way that you have some reason to think your judgment is better. But sometimes it’s the alligators! Most ideas for solving problems are bad, so your prior should be that if you have an idea, and it’s not being tried, probably the idea is bad; if you have inside-view reasons to think that it’s good, you should talk to the lifeguards to see if they’ve already considered this or think you will do harm.
Finally, it’s worth noting that even when the lifeguards are competent and correctly prioritizing, sometimes the job is just too hard for them to succeed with their current capabilities. Lots of top EAs are already working on AI alignment in not-obviously-misguided ways, but it turns out that it’s a very very very hard problem, and we need more great lifeguards! (This is not saying that you need to go to “lifeguard school,” i.e. getting the standard credentials and experiences before you start actually helping, but probably the way to start helping the lifeguards involves learning what the lifeguards think by reading them or talking to them so you can better understand how to help.)
(Even) More Early-Career EAs Should Try AI Safety Technical Research
Hmm, interesting. My first draft said “under 1,000” and I got lots of feedback that this was way too high. Taking a look at your count, I think many of these numbers are way too high. For example:
FHI AIS is listed at 34, when the entire FHI staff by my count is 59 and includes lots of philosophers and biosecurity people and the actual AI safety research group is 4, and counting GovAI (where I work this summer [though my opinions are of course my own] and is definitely not AI safety technical research).
MIRI is listed at 40, when their “research staff” page has 9 people.
CSET is listed at 5.8. Who at CSET does alignment technical research? CSET is a national security think-tank that focuses on AI risks, but is not explicitly longtermist, let alone a hub for technical alignment research!
CHAI is listed at 41, but their entire staff is 24, including visiting fellows and assistants.
Should I be persuaded by the Google Scholar label “AI Safety”? What percentage of their time do the listed researchers spend on alignment research, on average?
With “100-200” I really had FTEs in mind rather than the >1 serious alignment threshold (and maybe I should edit the post to reflect this). What do you think the FTE number is?
Thanks for these points, especially the last one, which I’ve now added to the intro section.
I support some people in the EA community taking big bets on electoral politics, but just to articulate some of the objections:
solving the “how to convince enough people to elect you president” problem is probably easier than a lot of other problems
Even compared to very difficult other problems, I’m not sure this is true; exactly one person is allowed to solve this problem every four years, and it’s an extremely crowded competition. (Both parties had to have two debate stages for their most recent competitive cycles, and in both cases someone who had been a famous public figure for decades won.)
And even if you fail to win, even moderately succeeding provides (via predictable media tendencies) a far larger platform to influence others to do Effective things.
It provides a larger platform, but politics is also an extremely epistemically adversarial arena: it is way more likely someone decides they hate EA ideas if an EA is running against a candidate they like. In some cases this trade-off is probably worth it; you might think that convincing a million people is worth tens of millions thinking you’re crazy. But sometimes the people who decide you’re crazy (and a threat to their preferred candidates) are going to be (e.g.) influential AI ethicists, which could make it much harder to influence certain decisions later.
So, just saying—it is very difficult and risky, so anyone considering working on this needs to plan carefully!
We’re working on making Boston a much better hub—stay tuned!
In addition to the biosecurity hub, advantages for Boston not listed in the Boston section include immediate proximity to two of the top 2/5/5 global universities (the only place on earth where two are within a mile of each other), an advantage both for outreach/community-building and for the “culture fit” aspects discussed in this post.
It’s also nearly ideally positioned between other EA hubs and mini-hubs:
Non-horrific distance in both time zone and flight to London (5 hours apart/6.5 hour flight) and San Francisco (3 hours apart/7 hour flight). Decent flight connectivity to Central Europe as well (though NYC is better for this).
Easy train ride to NYC (on which I am typing this comment!) and quick flights to NYC/DC.
Same time zone and 3.5 hour flight to Bahamas.