This post is part of the “Insights of an ERA: Existential Risk Research Talent Development” sequence, outlining lessons learned from running the Existential Risk Alliance (ERA) Fellowship programme in 2023.

Background

ERA is running an annual 8-week Summer Research Fellowship for junior researchers interested in existential risk. (For more information on this, see our Announcement and our Sequence including a Hiring Retrospective and our Theory of Change.)

This is the last of a series of posts where we—the ERA team—provide an insight into our program. In this post, we take a closer look at some of the work that previous CERI and ERA Fellows are doing nowadays. This post is not intended to provide proof of the impact that ERA has had—we are conducting a separate impact evaluation in-house and might publish it in the future. Rather, we realised that making it easier for people to understand what kind of career trajectories are pursued by the alumni of the program could help potential applicants understand their fit for the program.

We have also published some of our previous Fellow’s research on our website, and we encourage you to check it out! As more and more Fellows go through our program, we hope to build this out to be even more representative of ERA’s research focus.

Alumni Profile #1: Claire Dennis

What they did before ERA/CERI: Immediately before joining the fellowship, Claire graduated with her Master’s degree in Public Affairs from Princeton University, where she completed a two-year program focused on Science, Technology, and Environmental Policy. During her time at Princeton, Claire focused on climate change and AI policy, as well as their intersection. She took on a role as an Environmental Policy Associate, representing Princeton at the COP27 international climate conference in Egypt. Claire also studied technical machine learning and served as a Policy Fellow at the Responsible AI Institute.

What they worked on as Fellows: Leveraging her knowledge in global environmental governance and her experience working in public service and international organisations, Claire embarked on the ambitious journey of figuring out what role the UN can play in providing a pathway to international collaboration on AI Governance. Together with collaborators from the Simon Institute for Longterm Governance and the UN University, she co-authored a report titled “Towards a UN Role in Governing Foundation Artificial Intelligence Models,” emphasising the UN’s need to focus governance efforts on advanced AI systems. She was supported by her mentor Carlos Gutierrez, who works with the Future of Life Institute and the OECD. Having just pivoted into the field of Artificial Intelligence, Claire also spent some amount of her time at ERA to upskill and learn about different approaches in the space, network with like-minded researchers, and identify current opportunities that she is excited to work on in the future.

What they currently work on: Claire’s present focus is on cultivating her career and making a meaningful impact in the still nascent field of international AI governance. She is actively pursuing opportunities with the United Nations and the United States Government. Additionally, Claire is currently co-authoring a paper proposing an international governance framework aimed at mitigating the risks posed by the unconstrained development of advanced AI.

In their own words: “The ERA fellowship has served as a pivotal step in my career in international AI governance. It introduced me to a community of deeply passionate, intelligent individuals who are exploring what this incredibly powerful technology could mean for the world. It placed me in a small but growing group of people who are aware of its risks and who are advocating to prevent unnecessary large-scale harm. The timing could not have been better for me to join this fellowship and explore AI regulation on an international scale. The field of AI governance is taking off incredibly fast. In my 8 weeks in Cambridge it seemed like every day there were breaking headlines on the topic – the announcement of the UK AI Summit, the Frontier Model Forum, the US Senate hearings on AI risks, among many others. Being able to dissect and analyze technical concepts, current events, key actors, and potential future scenarios with a community of like-minded researchers was an opportunity I am incredibly grateful for. Ultimately, I used the opportunity provided by ERA to build professional connections and produce research that will undoubtedly serve as a bedrock for my continued growth in this dynamic field.”

Alumni Profile #2: Charlie Harrison

What they did before ERA/CERI: Prior to ERA, Charlie was fairly involved with the London EA Community. He was Co-President of the UCL EA Soc, and had been interested in AI, although he had not done much concentrated work in existential-risk-related fields. Charlie found out about ERA through other London EAs who had done CERI in previous years.

What they worked on as Fellows: As an ERA Fellow, Charlie looked at whether protestors could slow down AI progress, by investigating other historical examples of successful protests, particularly those against GMOs.

What they currently work on: Based on the work he conducted while being an ERA Fellow, Charlie is currently working on 3 forthcoming forum posts, some of which will also be submitted for publication:

Case studies of protests
A historical deep-dive into GM crop protests and lessons for AI protests
Is there a “crucial window” for AI protests?

Additionally, Charlie is looking into starting a podcast on AI.

In their own words: “ERA has opened many doors for me: I’ve met many prominent scholars working in this area, and eventually will have several research outputs. I’ve presented my work to several people and groups since the end of the Fellowship. I am unsure whether I will continue working in this area in the long-term, but working in existential risk seems like a much more viable route.”

Alumni Profil #3: Sarah Weiler

What they did before ERA/CERI: Before joining ERA as a Summer Research Fellow, Sarah was studying for her Master’s degree in International Relations and with a focus on China at a Danish university. She found out about the program through a conversation with one of the cause area leads at the time.

What they worked on as Fellows: During her time in Cambridge, Sarah worked on a project related to nuclear risk. Her write-up was published as a sequence on the EA Forum. Her main goal was to get an overview of different approaches in the field and to evaluate the different ideas that existed to mitigate existential risk from nuclear weapons.

What they currently work on: Sarah currently works as a data scientist for the European Commission’s Joint Research Centre with a special focus on internal conflict risks outside the European Union and early warning systems. After finishing her CERI project, she was also contacted by staff from ALLFED and CSER to work on a joint project related to nuclear risk reduction, which she says had a very low chance of happening if it wasn’t for her participation in the Fellowship.

In their own words: “CERI provided a rather unique setting in that it gave me ample opportunities to connect, challenge and be challenged by other dedicated and inspiring people. I’ve made connections that I otherwise couldn’t have made, and felt motivated by the environment and the people in it. My 10 weeks at CERI have had quite an influence on how I think about research and epistemology, but also ethics. I have also learnt to deal with challenges and how to synthesize information from different sources, plan a research project and improve my writing skills. This continues to influence how I do research nowadays.”

Alumni Profile #4: Catherine Brewer

What they did before ERA/CERI: Catherine joined ERA after their first year of studying PPE at Oxford. They previously worked as a research assistant at Oxford’s School of Geography and the Environment.

What they worked on as Fellows: Catherine worked within ERA’s AI governance stream, and carried out a project on understanding and predicting applications of AI to state surveillance. Their research focused on case studies of government adoption of surveillance technologies, and quantitative modelling of factors influencing government applications of AI to surveillance.

What they currently work on: Catherine just finished a summer research fellowship at GovAI, where they researched case studies of rapid US regulatory change. They’re now participating in MLAB and going into their final year of undergrad at Oxford, where they run the AI governance team of the student AI safety group.

In their own words: “ERA was a great opportunity to explore AI governance research, learn more about fields relevant to existential risk, and meet researchers and academics working on similar topics. My favourite part of ERA was by far the other fellows: it was amazing to get to work alongside such smart, dedicated, curious people, and it definitely changed how I think. I think I learned as much from conversations and informal reading groups/discussions as from the formal programming. I’d recommend ERA if you’re an early-career researcher or interested in trying out existential risk-related research, and think you’d benefit from meeting people, having space and time to think independently about a topic of your choice, and getting some mentorship on research skills.”

Alumni Profile #5: Iyngkarran Kumar

What they did before ERA/CERI: Before the fellowship, Iyngkarran had just finished an undergraduate degree in Natural Sciences, specialising in physics and computer science, at Durham University. He had spent most of his second and third year getting up to speed with AI existential risk, taking part in the AGISF technical track and organising a similar reading group at Durham. He also completed his undergraduate dissertation in neural network interpretability.

What they worked on as Fellows: As an AI Governance fellow, Iyngkarran worked on modelling cyber attacks from advanced AI systems, under the mentorship of a member of Rethink Priorities’ AI Governance and Strategy team. His worked involved elucidating the necessary steps that an AI system would have to execute in order to engage in dangerous cyber behaviour, as well as key bottlenecks to the occurrence of such events. In addition, he identified some high-level steps that labs and/or auditors could take to prevent damaging cyber attacks from advanced AI.

What they currently work on: Iyngkaran will be spending the next year working as a research assistant for the Transformative Futures Institute, working on forecasting the impacts of transformative AI systems and accelerating growth of the infrastructure required for evaluation of frontier models. Alongside this, he will be scoping and executing projects in the field of AI takeoff speeds, potentially in collaboration with other researchers in the field.

In their own words: “I strongly recommend the ERA fellowship to anyone who is exploring the area of existential risk, or students and early-career researchers who have already committed to the field. The networks developed during the fellowship through interactions with mentor, research manager and other fellows are invaluable, and it’s highly likely that I would not be working in my current position if not for the fellowship. I commend the management of the fellowship for providing fellows with a stimulating intellectual environment throughout the summer, and promoting growth of both the fellows’ knowledge of their cause area and the skills required to do effective research.”

Alumni Profile #6: Ximena

What they did before ERA/CERI: Ximena was studying and researching disaster risk during their undergraduate and their masters degree. They did their undergrad in Geography at the University of Cambridge, and they are currently finishing their masters in Risk Analysis at King’s College London.

What they worked on as Fellows: Ximena was looking into how disaster risk concepts could be useful for existential risk studies. They found that in existential risk studies, thinking about risk is overwhelmingly hazard-oriented, whereas in disaster risk there is more emphasis on how the hazard will affect vulnerable groups.

What they currently work on: Ximena is just about to finish their Masters in Risk Analysis at King’s College London, and hopes to pivot into a career reducing catastrophic climate risk either through research or governance

In their own words: “One of the best things I got out of ERA was meeting people who are very committed to the topics they are working on. Lots of people I know in London are stuck in careers they don’t seem to like very much. Being around people who are working on things that are really important to them improved my feeling that I can do something meaningful with my career. It was also a great opportunity to test my fit for existential risk research. Having discussions with people who hold different ideologies from you is always valuable, and I had many of these during ERA. Engaging with ideologies I disagree with helped me realise that xrisk is an area where I could actually make meaningful, novel contributions.”

Conclusion

We are very excited about seeing more and more of our Fellows take impactful research roles and advance the field of existential risk research. We feel it’s important to acknowledge that these are only a few examples of Fellows who have benefitted from the program and they should not be seen as a representative sample. Similarly, there are other alumni we are very excited about that we couldn’t include in this post to keep it at a readable length.

We also acknowledge that this is but a snapshot which, in order to draw valid conclusions about the impact of the program, we need to complement with ongoing analysis and tracking of alumnis’ careers. We hope to be able to collect more and more data on this as we go through additional cohorts.

ERA Fellowship Alumni Stories