Nuclear risk research ideas: Summary & introduction

MichaelA🔸8 Apr 2022 11:17 UTC
103 points
4 comments7 min readEA link

Note: I haven’t put as much time into this series of posts as I’d like, but thought it’s better to post them in their current form than never post at all. Also, I mostly wrote these posts in late 2021, and I haven’t attempted to update them in light of (a) the 2022 Russian invasion of Ukraine or (b) the New Nuclear Security Grantmaking Programme at Longview Philanthropy. I’d now encourage readers to seriously consider applying to Longview’s roles (if they’re still open when you read this) and to have Longview in mind as a key potential user/​hirer/​ally for nuclear risk work readers might do.[1]

Summary & the list of ideas

This series of posts outlines possible research projects that I think would be tractable and could substantially help us work out (a) how much to prioritize nuclear risk reduction relative to other important problems and (b) what interventions to prioritize within the area of nuclear risk reduction. There are also various forms of support that may be available to someone interested in pursuing these projects, which I overview later in this introductory post.

Each project idea should make sense by itself, so you should feel free to read only this post and then the post(s) for any particular idea(s) you’re interested in.[2] For most of the project ideas, I briefly discuss how the project could be tackled, why it might be useful, what sort of person might be a good fit for it, and whether people in the effective altruism community (“EAs”[3]) should try to fund/​convince people outside the EA community (“non-EAs”) to do this project.[4] I also list previous work that could be worth reading and people that could be worth talking to.[5]

What follows is a table listing the main project ideas in this series along with my rough, subjective, low-confidence bottom-line views about how important, tractable, neglected, and “outsourceable” each idea is. The tractability scores take into account the abandoned drafts or notes that I know other researchers (such as me) have already written on some of these topics and that could be used as a starting point by a new researcher. The neglectedness scores take into account work that’s already been done, is in progress, or seems likely to be done by default. The outsourceability scores are intended to capture how well I imagine funding/​convincing non-EAs to do a given project would work.

But this table doesn’t capture the person-specific considerations of personal fit, testing fit, or building career capital; as such, you should think for yourself about those considerations rather than blindly following this table’s ratings of each idea. I also haven’t tried to capture how long each project would take, partly because each project could take many different forms and levels of extensiveness.

The table links to posts on each of the individual ideas.

Project ideaImportanceTractabilityNeglectednessOutsourceability
Intermediate goals for nuclear risk reductionHighMediumMedium/​HighMedium
Technological developments that could increase risks from nuclear weaponsMedium/​HighMediumMediumMedium
Climate, agricultural, and famine effects of nuclear conflictHighMedium/​LowMediumMedium/​High
How should EAs react to funders pulling out of the nuclear risk space?MediumMedium/​HighMedium/​LowMedium
Impact assessment of nuclear-risk-related orgs, programmes, movements, etc.MediumMediumMediumMedium
Overview of nuclear-risk-related projects and stakeholdersMedium/​LowHighMedium/​LowMedium
Nuclear EMPsMediumMediumMediumMedium
Polling or message testing related to nuclear risk reduction and relevant goals/​interventionsMediumMediumMediumMedium/​High
Neartermist cost-effectiveness analysis of nuclear risk reductionMedium/​LowMedium/​HighMedium/​LowLow
Direct and indirect effects of nuclear falloutMedium/​LowMediumMedium/​LowMedium
How bad would the worst plausible nuclear conflict scenarios be?Medium/​LowMediumMedium/​LowLow


There’s also an Appendix containing additional ideas that didn’t make it into the post, either simply because I ran out of time to look into or explain them properly or because I tentatively believe they’re lower priority.

Background for this set of ideas

I’m very unsure how many people and how much funding the effective altruism community should be allocating to nuclear risk reduction or related research, and I think it’s plausible we should be spending either substantially more or substantially less labor and funding on this cause than we currently are (see also Aird & Aldred, 2022a).[6] And I have a similar level of uncertainty about what “intermediate goals”[7] and interventions to prioritize—or actively avoid—within the area of nuclear risk reduction (see Aird & Aldred, 2022b). This is despite me having spent approximately half my time from late 2020 to late 2021 on research intended to answer these questions, which is—unfortunately! - enough to make me probably among the 5-20 members of the EA community with the best-informed views on those questions.

And while doing that research, I’ve collected or generated many ideas for further research projects that I think could help us make substantial progress towards better understanding how much to prioritize nuclear risk reduction and/​or what to prioritize within this area. These are projects I’d have been keen to do if I had the time or the relevant skills[8] and that I’m keen for someone else to do.

Support that people who want to work on these projects may be able to get

(See also my Notes on EA-related research, writing, testing fit, learning, and the Forum. Also note that the following options are neither exhaustive nor mutually exclusive.)

  • Support via being part of an EA-aligned research training program (see this list)

  • Grants

  • Notes, advice, feedback, or connections from me and/​or from Will Aldred (who collaborated with me on some nuclear risk research and leads EA Cambridge’s nuclear risk community building)

    • Will and I can’t make any guarantees, partly because we don’t know what our other commitments will be in future. But each of us are potentially open to:

      • Having a call with someone who’s strongly considering working on one of these topics to discuss the idea further, suggest more readings and people to talk to, suggest research training programs or job opportunities, etc.

      • Making introductions to potential feedback-givers, mentors, or users of eventual research outputs

      • Sharing relevant unpolished research notes

      • Giving feedback on one or more drafts

    • If you’re interested in that, I suggest emailing both Will and I in the same thread at will.aldred [at] eacambridge [dot] org and michaeljamesaird [at] gmail [dot] com

  • Notes, advice, feedback, or mentorship from other people (in or outside of the EA community)

  • Non-EA degree programmes or jobs

    • E.g., Masters programmes, PhD programmes, or positions at think tanks who do relevant work

  • Research roles at some organizations which do work related to longtermism/​existential risk (see also 80,000 Hours’ job board)

Caveats

  • This post expresses my personal opinions, not necessarily those of any of my current or former employers.

  • I don’t simply mean to advocate for more people to focus on nuclear risk reduction or on research rather than other longtermism-aligned cause areas or career paths. There are many other cause areas, research projects (see Aird, 2020), and career paths that are also important and will be better fits for many people. (See also footnote 5.)

  • I’m still far from an expert on nuclear risk myself, so I expect that some of these research projects are lower priority than I think,[9] that some of my suggestions for how to pursue them are far from ideal, and that I’ve overlooked several other high priority research project ideas.

  • I of course can’t guarantee that a given person will be able to get the kind of support mentioned in the above section

Some general points about theories of change and methods

These project ideas vary in their theories of change and the best methods for tackling them. However, I think the following breakdown of broad, abstract, not-mutually-exclusive theories of change would often be useful to have in mind:[10]

  • Advancing the frontiers of the EA community’s knowledge: Providing the EA community with conclusions, information, concepts, etc. that even the community members who are most informed on the relevant topics weren’t aware of.

    • This may involve advancing the frontiers of humanity’s knowledge on these topics, but also may not (e.g., if there are relevant bodies of work that no EA community member has engaged with or extracted the relevant insights from).

  • Bringing more EAs to the frontiers of the EA community’s knowledge: Providing certain conclusions, information, concepts, etc., to a broader set of EA community members.

  • Advancing the frontiers of non-EAs’ knowledge and/​or bringing more non-EAs to that frontier

    • E.g., helping governments, non-EA researchers, or non-EA philanthropists decide what policies, questions, campaigns, etc., to look into or support.

(Of course, other breakdowns of the possible theories of change for these projects would also be possible and could be more useful.)

Relatedly, it also seems worth bearing in mind that many of these projects could be tackled “simply” by finding, aggregating/​summarizing, analyzing, and/​or drawing inferences from existing work or experts’ views,[11] such as via:

  • Just finding one piece of existing work that turns out to already tackle the topic quite well and highlighting this to relevant decision-makers or summarizing its implications for them

  • Summarizing, red-teaming, or working out the nuclear risk /​ longtermist implications of bodies of relevant existing work

  • Brief expert elicitation, such as having email exchanges or calls regarding the topic or sending out simple surveys, then summarizing takeaways[12]

Other potentially “low-effort” methods that could be used for these projects include:

  • Gathering relevant quantitative data or reasonable estimates and plugging them into Fermi estimates and/​or more carefully constructed quantitative models

  • Writing forecasting questions that are well-specified yet still would be informative for the project (or working with others to do so), and then making forecasts on such questions and/​or soliciting forecasts on them from other people (e.g., by putting the question on Metaculus)

This variety of possible theories of change and methods is part of why it’s hard to say how long a given project is likely to take; it depends on what approach a given person aims to take to the project.

Acknowledgements

My work on this series of posts was supported by Rethink Priorities. However, I ended up pivoting away from nuclear risk research before properly finishing the posts I was writing, so I ended up publishing this in a personal capacity and without having time to ensure it reached Rethink Priorities’ usual quality standards.

I’m very grateful to Will Aldred for a heroic bout of editing work to ensure my rough drafts finally made it to publication. I’m also grateful to David Denkenberger, Dewi Erwan, Jeffrey Ladish, and Linch Zhang, Luisa Rodriguez, and Peter Wildeford for helpful discussion or feedback on drafts of this post or on my earlier lists of nuclear risk research project ideas. Mistakes are my own.

  1. ^

    Disclaimer: Rethink Priorities, where I work, has received some funding from Longview Philanthropy. However, I’m confident I would’ve in any case believed it makes sense to encourage readers of this post to consider applying to Longview’s roles and considering Longview as a potential user/​hirer/​ally. This is because (1) Longview might (to my knowledge) currently be the effective-altruism-aligned funder with the greatest degree of focus on nuclear risk issues, and (2) one of the main conclusions I formed from my 2021 nuclear risk research was that an EA funder such as Longview should hire a grantmaker focused on nuclear risk issues, and I formed that view before learning Longview intended to do this.

  2. ^

    In reality, many of these “project ideas” are really more like broad areas, directions, or umbrellas that could contain many possible projects or specific directions within them.

  3. ^

    I use the terms “EAs” and “non-EAs” as shorthands for convenience. I don’t mean to imply that all people in those groups would identify with those labels or that there’s a sharp distinction between those groups.

  4. ^
  5. ^

    Specifically, I list work that could be worth reading other than just the work already cited in the section on the project in question. Likewise, I list people that could be worth talking to other than just me or authors of the works cited in the section.

  6. ^

    By “allocated to nuclear risk reduction or related research”, I mean allocated to direct or indirect nuclear risk reduction work, work to figure out how much to prioritize this area, work to figure out what to prioritize within this area, or work aimed at supporting any of those other types of work. I’m not including work that’s superficially about nuclear risk but is primarily intended to achieve other goals, such as working on nuclear risk primarily to build skills for later doing AI governance work (on that topic, see Aird, 2022)

    For what it’s worth, my 90% subjective confidence intervals are that the EA community, at its current size, should probably be allocating somewhere between 2 and 1000 community members and somewhere between $200,000 and $100 million per year to nuclear risk reduction or related research. But those are of course very wide ranges!

    These ranges are my 90% subjective confidence intervals for what my best guess would be after (1) a total of a year of full-time research was done on some of the topics I list in this post by people who are a good fit for them, and then (2) I carefully read and think about the outputs from that. These subjective confidence intervals aren’t completely pulled out of nowhere—I spent about 90 minutes trying to work out reasonable intervals, and have thought about similar questions for the past year—but they still feel fairly made-up and unstable.

    For information on the current size of the effective altruism community, see Todd (2021).

    Here I’ve focused only on the time and money of EA community members, since those two resources seem especially important and relatively easy to think about. But one could also consider other resources, such as political capital.

  7. ^

    By an intermediate goal, I mean a goal that (1) is more specific and directly actionable than a goal like “reduce nuclear risk”, (2) is of interest because advancing it might be one way to advance a higher-level goal like that, but (3) is less specific and directly actionable than a particular intervention (e.g., “advocate for the US and Russia to renew the INF Treaty”).

  8. ^

    The single biggest reason why I won’t end up doing these projects myself is that Rethink Priorities (where I work) was offered a large grant to build a large AI governance team, and my manager and I agreed that it made sense for me to pivot to helping manage that team.

    Additional reasons why I won’t end up doing these nuclear risk projects myself include that I’m just one person and that I lack some relevant bodies of knowledge, skills, and connections (e.g., in national security, international relations, or climate modeling).

  9. ^

    For example, some projects may have already been sufficiently addressed in existing work that I’m not aware of, or their answers may be sufficiently “obvious” to relevant decision-makers but just not to me.

  10. ^

    See also Aird (2021).

  11. ^

    See also Olah & Carter (2017).

  12. ^

    See also “Action: Conduct expert interviews” and my “Readings and notes on how to get useful input from busy people”.

MichaelA🔸8 Apr 2022 11:17 UTC
103 points
4 comments7 min readEA link