I’m a generalist and open sourcerer that does a bit of everything, but perhaps nothing particularly well. I’m currently the AI Safety Group Support Lead at CEA.
I was previously a Software Engineer in the Worldview Investigations Team at Rethink Priorities.
This is one of the most exciting forecasting proposals I’ve read in some time. I’m thoroughly impressed, and I’m really eager to see more work building on top of this.
If someone ends up running an algorithmic forecasting tournament like the ones proposed in this article, count me in ;)
I’ll note that I think that choosing to prioritize resources (or support) for top universities doesn’t imply that one thinks that EAs need to be any less altruistic or truth-seeking. One can prioritize groups at top universities while (for example) maintaining the same threshold for what a good organizer or group member looks like. These kinds of strategies might result in less diversity overall, but I don’t think they result in having less altruistic people.
Could you clarify what you mean by margin-based reasoning in this context?
I agree that it’s important to consider both needs and interests. Ultimately, a branding strategy should be embedded in a larger theory of change and strategy for your group, and that should determine which audiences you reach out to.
Regarding the latter, I agree that an interest in, say, hacker culture, does not adequately describe all people interested in CS. It might actually leave out a bunch of people that you should care about joining our group. At the same time, branding is all about tradeoffs, and you have to pick which things you cater to. Spread too thin, and you risk making the content too unappealing.
It’s hard to give empirical data on this because I don’t think we have a good track record of actually collecting it. I would be curious about groups trying things like A/B tests to refine their strategies.
So yeah, most of this is backed by a mix of anecdotes from organizers, marketing know-how, and some of what I learned running my old AI Safety Initiative. This is why I want to emphasize that most of this is to be taken as provisional rather than the final word on the matter.
A few years ago, I started an EA university group in Chile, got funding and support through UGAP, then got advising through 80k and continued mentoring through OSP. This year, I got hired at CEA’s Groups team.
It’s difficult to trace counterfactual impact, but I suspect this wouldn’t have been possible if UGAP hadn’t helped me get a group started, or if CEA wasn’t able to recognize good talent outside top universities.
I’m biased now, but I think people miss how most of our team’s efforts go to scalable support, which doesn’t necessarily target top universities (and includes universities like mine). Selecting top universities is a good heuristic for getting high-quality group members, but you also get diminishing returns fast because you’re bottlenecked by having great organizers at these universities who are in need of support. This, alongside the fact that there is such a thing as great groups outside of top universities, is why we still spend a lot of time, resources, and money on other universities worldwide.
I think this is fine: Epoch’s work appeals to a broad audience, and Nat Friedman is a well-respected technologist.
I read your post while I was writing up the wiki article on Shapley values and thought it was really useful. Thanks for making that post!
Quick poll [✅ / ❌]: Do you feel like you don’t have a good grasp of Shapley values, despite wanting to?
(Context for after voting: I’m trying to figure out if more explainers of this would be helpful. I still feel confused about some of its implications, despite having spent significant time trying to understand it)
Can anyone who is more informed on NIST comment on whether high-quality comments tend to be taken into account? Are drafts open for comments often revised substantially in this way?
This is just a top-notch post. I love to see detailed analyses like this. Props.
Hope to see you next year! 🤝
At last, a biblically accurate Qualy The Lightbulb
[Opinion exclusively my own]
I think this framing has a lot of value. Funnily enough, I’ve heard tales of groups like this from the early days of EA groups, when people were just figuring things out, and this pattern would sometimes pop up.
I do want to push back a little bit on this:
But before deferring, I think it’s important to realize that you’re deferring, and to make sure that you understand and trust who or what you’re deferring to (and perhaps to first have an independent impression). Many intro fellowship curricula (eg the EA handbook) come across more as manifestos than research agendas—and are often presented as an ideology, not as evidence that we can use to make progress on our research question.
The EA handbook (which nowadays is what the vast majority of groups use for their intro fellowships) includes three “principles-first” weeks (weeks 1, 2, and 7), which are meant to help participants develop their own opinions with the help of only the basic EA tools or concepts.
Furthermore, week 7 (“What do you think”) includes a reading of independent impressions, and learning that concept (and discussing where EA might be wrong) is one of the key objectives of the week:
A key concept for this session is the importance of forming independent impressions. In the long run, you’re likely to gain a deeper understanding of important issues if you think through the arguments for yourself. But (since you can’t reason through everything) it can still sometimes make sense to defer to others when you’re making decisions.
In the past, a lot of work has been put in trying to calibrate how “principles-based” or “cause-oriented” intro fellowships should be, and I think the tradeoff can be steep for university groups since people can get rapidly disenchanted by abstract philosophical discussion about cause prioritization (as you mention). This can also lead to people treating EA as a purely intellectual exercise, instead of thinking of concrete ways in which EA ideas should (for example) change their career plans.
That said, I think there are other ways in which we could push groups further along in this direction, for example:
We could create programming (like fellowships or workshops) around cause prioritization, exploring different frameworks and tools in the field. Not just giving a taste of these frameworks (like the handbook does), but also teaching hands-on skills that participants can use for actual cause prioritization research.
We could push for more discussion centered around exploratory cause research, for example, by creating socials or events in which participants try to come up with cause candidates and do some preliminary research on how promising they are (i.e. a “cause exploration hackathon”).
I know there has been some previous work in this direction. For example, there’s this workshop template, or this fellowship on global priorities research. But I think we don’t yet have a killer demo, and I would be excited about groups experimenting with something like this.
I’m so excited about the risk-based portfolio builder!