I’m a generalist and open sourcerer that does a bit of everything, but perhaps nothing particularly well. I’m currently the AI Safety Group Support Lead at CEA.
I was previously a Software Engineer in the Worldview Investigations Team at Rethink Priorities.
I couldn’t complete the exit survey for EAGx Latam due to a technical glitch, but just to add a little anecdotal evidence to its counterfactual impact:
Meetups for attendees from Mexico, Chile/Argentina, Colombia/Perú, Brazil and Spain.
As one of the student community builders from Chile, one of the most surprising things about EAGx LatAm was that it was the first time we met EA professionals from Chile. Most of them were people that we would probably have never found if we had not gone to the conference.[1]
Thanks to this, we were able to start a national/city group just from the sheer amount of Chilean attendees to the conference, making it the third national group to emerge in Latin America and perhaps (sorry Mexico) the one with the biggest number of professionals.
We even met Rob Gledhill (the head of groups at CEA) for the first time, even though he lived in the same city as us. Rob, if you’re reading this, I’m still sorry for abandoning you at the fountain 😅
I also got the same impression. He talks about “ethics”, but in context he seems (I’m not completely sure) to be talking about something like ethical injunctions specifically.
Some random appreciations (because someone nudged me to give positive feedback that I would have otherwise not shared with anyone):
I think the Online team at CEA has done an outstanding job improving the forum. Every month or so, they keep surprising me with more features I didn’t know I needed, which reflects them doing a lot of hard work behind the scenes to create as much value for the EA community as possible. I’ve also been able to observe firsthand them gathering user feedback and doing feature prioritization, and as a software engineer, I’m extremely impressed!
Even though the Community Health at CEA has made many errors in the past, I think we often miss both how difficult their work is, and how every decision they make is ridden with crucial tradeoffs, many of which are invisible to others[1]. Because every successful incident they handle correctly (or even prevent) is one incident we probably don’t hear about, I want to take a moment to highlight that in every interaction I’ve had with them, they’ve acted with incredible professionalism, demonstrated deep concern for everyone involved, and acted very competently.
Having been in similar positions many times as a community builder (both inside and outside EA), I know just how difficult the job is, and how, for example, often what others see as clear failures are just the result of them lacking information which can’t be publicly shared without harming others.
I strongly agree.
I miss discussions about “how we can make EA mainstream” or “bring EA to academia”.
While I find the EA community to be a great source of personal and social value, we still face the challenge of significantly scaling everything we do. Taking informed steps to doing good better shouldn’t be a side consideration for governments, NGOs or people, it should be the default. Working to systematically address existential risk shouldn’t be the work of a few nonprofits, it should be the work of national and international institutions.
If we over-emphasize community building while at the same time de-prioritizing engaging with the outside world, we risk that vision. There are significant advantages to working inside a community (and we should leverage those!), but to be truly successful, we first have to learn how to communicate with the outside world.
For group organizers, one step we could take is prioritize working with existing institutions and experts in different fields. Instead of only inviting EAs to events, we could invite more experts working in related areas. Instead of only networking with EA institutions, we could work more closely with traditional institutions from different fields. [1]
This probably means navigating difficult tradeoffs, in handling outreach and press, in compromising ideas and in producing less “highly-engaged EAs”, but I think this is a discussion worth having.
This is already happening in the policy space (out of necessity). There’s also plenty of precedent from other EA groups (especially if they’re specialized), but I don’t think it’s nowhere there yet.
Hi Michael! Some answers:
2. Is there a way to increase the sample size? It’s 150,000 by default, and you say it takes billions of samples to see the dominance of x-risk work.
There will be! We hope to release an update in the following days, implementing the ability to change the sample size, and allowing billions of samples. This was tricky because it required some optimizations on our end.
3. Only going 1000 years into the future seems extremely short for x-risk interventions by default if we’re seriously entertaining expectational total utilitarianism and longtermism. It also seems several times too long for the “common-sense” case for x-risk reduction.
We were divided on selecting a reasonable default here, and I agree that a shorter default might be more reasonable for the latter case. This was more of a compromise solution, but I think we could pick either perspective and stick with it for the defaults.
That said, I want to emphasize that all default assumptions in CCM should be taken lightly, as we were focused on making a general tool, instead of refining (or agreeing upon) our own particular assumptions.
5. It seems the AI Misalignment Megaproject is more likely to fail (with the same probability of backfire conditional on failing) than the Small-scale AI Misalignment Project. Why is that? I would expect a lower chance of doing nothing, but a higher chance of success and a higher chance of backfire.
As with (3), I agree with your reasoning, and we’ll probably be updating some of these template projects soon, but I would encourage you to tweak these assumptions to match yours.
Latin American students seem (internally) to feel really pressured to do a master’s and/or PhD abroad (e.g. US)
I feel attacked by this (and I’m only like 80% kidding).
I guess this is partially a by-product of not seeing many ways to gain legitimacy outside getting a postgraduate degree elsewhere. EA is somewhat different in the sense that people seem to weight academic degrees less heavily, but outside of EA this is more or less true: even if you come from a highly ranked university in Latin America, getting a job outside of it is quite hard, and most people will implicitly or explicitly discriminate against you.
In general, your observations seem especially helpful for the “Carreras con Impacto” initiative being run by the Spanish-speaking community. I think we could probably brainstorm ways to directly address these when advising undergraduates.
I would also give a big +1 to these observations:
Latin American students seem to have a hard time thinking ambitiously, probably more so than the “average EA”.
When these students come from low/middle income backgrounds then the issue seems to be that they are carrying a lot of “baggage” from their past that makes it hard for them to “think big” about the impact they can have in the future
If they come from more high income backgrounds then the issue seems to rather be imposter syndrome.
These are things we had to very explicitly address when working in community building in Chile, and I’m pretty sure other groups in Latin America have had similar problems. I think having good role models from the Spanish-speaking community have really helped ameliorate the lack of ambitiousness, but we still don’t have a clear-cut strategy for how to deal with it.
Thanks for this post!
I feel like EAGs applications are by default relatively short (≤ 2 hours), and as a general principle it seems good that CEA is actually asking for more information when needed (I didn’t even know they did this).
As for whether the second request might be unreasonable, it seems like we don’t have enough context on the project, your application, or the specific CEA email to make any strong judgements on this.
I’m not going to waste my time and theirs on endless rounds of edits, and given that the application process is probably the simplest part of organizing a conference, this leads me to expect a poorly put together event even if they were willing to admit me with my current answers.
What I’m surprised about is that at this point it doesn’t seem like you’ve had “endless rounds of edits”, but rather just two email exchanges. This feels uncharitable. Perhaps someone in the organizing team screwed up, so my natural response would be to ask for clarification if I felt the request was vague or unexpected?
I understand applications to EAGs can be burdensome and frustrating, but this also applies to the team reviewing them. Applications are not “the simplest part of organizing a conference”. They’re challenging to get right, especially at scale.
While I agree these contradictions are worth exploring, I don’t think (1) directly contradicts the incompetence hypothesis. Seeing the article:
Meanwhile, at a meeting with Alameda employees on Wednesday, Ms. Ellison explained what had caused the collapse, according to a person familiar with the matter. Her voice shaking, she apologized, saying she had let the group down. Over recent months, she said, Alameda had taken out loans and used the money to make venture capital investments, among other expenditures.
Around the time the crypto market crashed this spring, Ms. Ellison explained, lenders moved to recall those loans, the person familiar with the meeting said. But the funds that Alameda had spent were no longer easily available, so the company used FTX customer funds to make the payments. Besides her and Mr. Bankman-Fried, she said, two other people knew about the arrangement: Mr. Singh and Mr. Wang.
I read this as completely coherent with what SBF said: “each step was in isolation rational and reasonable, and then when I finally added it all up last week it wasn’t”.
As for (2), I don’t think it’s a big update on the fraud hypothesis. A backdoor can mean having admin access to a server, which I think gives a more parsimonious explanation.
It’s worth pointing out that if SBF was malicious and fraudulent, this whole interview could have just been an attempt to save faces, which is why we shouldn’t update strongly either way. We should wait till more external evidence arrives.
One problem I have with the methodology employed in this paper is that it’s fundamentally testing whether there is an increase in risk when researchers get access to LLMs, and not non-experts.
In practice, what I care about is how the risk increases when actors with virtually no expertise (but a lot of resources) are assisted by LLMs. Why? Because we’ve had resourceful actors try this in the past, particularly Al Qaeda in 2001.
Edit: As Lizka pointed out, they did test with two groups with no bio experience, but they didn’t have a control group. The study still provides useful data points in this direction.
I can second this: CEA’s retreats are top tier
Yes! We plan to release the source code soon.
I just wanted to say this is great!
Being transparent about events like these help address many concerns EAs have had in the past regarding events that might seem “secretive” at first glance, so I very much welcome this.