Haydn has been a Research Associate and Academic Project Manager at the University of Cambridge’s Centre for the Study of Existential Risk since Jan 2017.
HaydnBelfield
Karma: 3,751
Yes of course! KSR hinted there may be some interest in Ministry—Mars seems stuck in development hell unfortunately.
I didn’t mean to imply that you were plagiarising Neel. I more wanted to point out that that many reasonable people (see also Carl Shulman’s podcast) are pointing out that the existential risk argument can go through without the longtermism argument.
I posted the graphic below on twitter back in Nov. These three communities & sets of ideas overlap a lot and I think reinforce one another, but they are intellectually & practically separable, and there are people in each section doing great work. Just because someone is in one section doesn’t mean they have to be, or are, committed to others.
See also Neel Nanda’s recent Simplify EA Pitches to “Holy Shit, X-Risk”.
I think this is a very cool idea!
To offer some examples of similar things that I’ve been involved in—the trigger has often been some new regulatory or legislative process.
“woah the EU is going to regulate for AI safety … we should get some people together to work out how this could be helpful/harmful, whether/how to nudge, what to say, and whether we need someone full-time on this” → here
“woah the US (NIST) is going to regulate for AI safety...” → here
“woah the UK wants to have a new Resilience Strategy...” → here
“woah the UK wants to set up a UK ARPA...” → here
“woah the UN is redoing the Sendai Framework for Disaster Risk Reduction? It would be cool to get existential risk in that” → here, from Clarissa Rios Rojas
This is the kind of reactive, cross-organisational, quick response you’re talking about. At the moment, this is done mostly through informal, trusted networks. Could be good to expand this, have a bigger set of people willing to jump in to help on various topics. The list seems most promising on that regard.
Other organisations:
CSET was in some ways a response to “woah the conversation around AI in DC is terrible and ill-informed”—a kind of emergency response.
FLI have been good at taking advantage of critical junctures through e.g. their huge Open Letters.
ALLFED has a rapid response capability, they wrote about it here. Having a plan, triaging, and bringing in volunteers seem like sensible steps.
Some of the monitoring work being done full-time (not by volunteers) in DC, London and Brussels seems especially useful for raising the alert to others.
Finally, CSER’s Lara Mani has been doing some really cool stuff around scenario exercises and rapid response—like this workshop. For example, she went to Saint Vincent to help with the evaluation of their response to the eruption of La Soufrière (linked to her work on volcanic GCR). She also co-wrote: When It Strikes, Are We Ready? Lessons Identified at the 7th Planetary Defense Conference in Preparing for a Near-Earth Object Impact Scenario. Basically, I think exercises could be really useful too.
This is a side-note, but I dislike the EA jargon terms hinge/hingey/hinginess and think we should use the term “critical juncture” and “criticalness” instead. This is the common term used in political science, international relations and other social sciences. Its better theorised and empirically backed than “hingey”, doesn’t sound silly, and is more legible to a wider community.
Critical Junctures—Oxford Handbooks Online
The Study of Critical Junctures—JSTOR
https://users.ox.ac.uk/~ssfc0073/Writings%20pdf/Critical%20Junctures%20Ox%20HB%20final.pdf
https://en.wikipedia.org/wiki/Critical_juncture_theory
I just linked to that too! I think about it all the time.
I think your assessment of the lack of diversity in EA is right, that this is a problem (we’re missing out on talented people, coalition allies, specific knowledge, new ideas, wider perspectives, etc), and that we need to working towards improving this situation. On all three (questions 1-3), see this statement from CEA. Thanks for raising this!
In terms of what we can be doing, being inclusive in hiring and pipeline-building seem very important—Open Philanthropy are amongst the best practice on this (see here) and Magnify Mentoring are doing awesome work.
Empowering marginalised/affected communities directly and working closely with them is one of the reasons GiveDirectly is great and is strongly supported by the EA community. This can’t work so clearly with farmed animals and future people, of course.Just to add to other links people have offered, I’ve always liked this on privilege, and this discussion:
Building on pioneering work on ‘retrocausality’ by Huw Price in Time’s Arrow and Archimedes Point: New Directions for the Physics of Time.
This is really awesome! Well done for launching this, very excited to see what you achieve
I know lots of people who are incredibly impactful and are parents and/or work in academia. For many, career choices such as academia are a good route to impact. For many, having children is a core part of leading a good life for them and (to take a very narrow lens) is instrumentally important to their productivity
So I find those claims false, and find it very odd to describe those choices as “concession[s] to other selfish or altruistic goals”. We shouldn’t be implying “maximising your impact (and by implication being a good EA) is hard to make compatible with having a kid”—that’s a good way to be a tiny, weird and shrinking niche group. I found that bullet point particularly jarring and off-putting (and imagine many others would also) - especially as I work in academia and am considering having a child. This was a shame as much of the rest of the post was very useful and interesting.
Online participation is open. Its a hybrid format so some people will be there in person—but we’re at in-person capacity.
This a really useful set of thoughts and further readings, thanks for sharing Seth. I especially liked your points on policy windows, and on cross-fertilisation between the nuclear and GCR fields.
Shared the Twitter thread below on Feb 28, crossposting as similar reflections:
A few thoughts on global catastrophic risk a few days into Putin’s invasion—would be very interested in other’s reflections!
Nukes. Still the most urgent risk. Probably the most dangerous situation since 1983? Crucial to avoid misperceptions/mistakes
Networks. War is having ripple effects on grain/Lebanon, on air freight, obviously finance. Highlights importance of Classifying GCR (Avin et al) type work—eg on volcanoes (Mani et al)
Norms matter. ‘No wars of conquest’ does really seem to have held back great powers, and states willing to spend a lot to uphold that norm -EU a more consequential actor than many may have thought—Russia may be less important going forward if its high-tech industry craters
Great power tension/conflict and authoritarianism increase our GCR exposure (Liu et al) / are very important risk factors (Ord)
We should take risk-superior moves (Wiener) - moving to nuclear power and renewables important for both climate change and for reducing the above risk factors, etc
Really excellent that you spotted this gap in the philanthropic market and moved in to fill it. Well done! Hope you hire someone excellent.
Just briefly on (4) - Govts of all parties oppose all PMBs as a matter of course, especially ones from the Lords. Very few actually become law (see eg here). This pattern is less due to the specifics of any particular Bill, and more about govt control of the parliamentary timetable, and govts’ ability to claim credit for legislation. One’s options if one comes top of the PMB ballot is to 1) try and get the Govt to support it or 2) use it as a campaigning device (or I guess 3 try both).
I’m not so sure that the ideas in this Bill couldn’t get picked up by Conservatives—its introduced in the Commons by a Conservative MP, the well-being goals seem like the levelling up goals, extending the Risk Register is just sensible, NAO & OBR were both introduced by Conservative governments, etc. You’ll know the Conservatives better than me though—I liked your suggestions in the second half about ideas that might be more amenable.
Hi Larks and John, Thanks for sharing this with me ahead of posting.
Five notes for readers.
1.
First, this Bill isn’t an EA Bill. This is recognised a bit in the post, but I really want to underline it. Its led by Lord John Bird and his office, and supported by Today for Tomorrow. It mostly builds on the Welsh Commissioner for Future Generations. None or them are ‘EA’. There are about 3-4 supporters that could plausibly be labelled EA, out of ~100 institutional supporters.
2.
Second, on the merits of the Bill—to add a little to Sam’s excellent overview. Some useful further readings:
The independent House of Lords library produced 2 useful briefings:
https://lordslibrary.parliament.uk/wellbeing-of-future-generations-bill-hl/
https://lordslibrary.parliament.uk/research-briefings/lln-2019-0076/Great CSER overview of several forms of institutional representation: https://www.cser.ac.uk/resources/representation-future-generations/
The text of the 2015 Welsh Act: https://www.futuregenerations.wales/about-us/future-generations-act/
An assessment of how the 2015 Act has gone in Wales over the last 7 years, from one of its architects: https://www.futuregenerations.wales/resources_posts/futuregen-lessons-from-a-small-country/
A more independent assessment: https://link.springer.com/book/10.1007/978-3-030-02230-3
Overview of many longtermist institutions: https://globalprioritiesinstitute.org/wp-content/uploads/Tyler-M-John-and-William-MacAskill_Longtermist-institutional-reform.pdf
3.
Third, I think the Bill is mainly about ‘broad longtermism’ https://80000hours.org/podcast/episodes/ben-todd-on-varieties-of-longtermism/
I think the most important parts of the bill are about longtermist representation, rather than big welfare-affecting policies. For example, the parliamentary committee on future generations, an independent commissioner, the responsibility on ministers, the NAO/OBR oversight, the longer Risk Register timeframe, the “set some longterm goals”/impact assessments—everything seems designed to just nudge politicians to think more about future generations.
The idea, I presume, is that all that procedural nudging (without being specific about substance, which should be left to current elected politicians) will prompt more long-term thinking and move away from our incredible shorttermism (eg see some of the stuff Cummings has written about how incredibly shortterm our political culture is).
4.
Fourth, the authors note “Some supporters seem to think most of the chance for this or a similar bill being passed rests on a future Labour government, but this may not happen for many years”
I presume this is just based on me (as I said in comments on the draft). However, that’s not my view. I think both parties could (and should) support it—its being led in the Commons by a Conservative. I don’t think it shold be coded as Labour, and if I tweeted stuff that may have given that impression, I regret that. I do think there’s a slightly higher chance of it being passed by a Labour govt, but mainly because of the Welsh link—most Welsh MPs are Labour.
5.
Fifth and finally, I find it helpful to return to the Bill’s overview, as laid out by Lord John Bird in the explanatory notes. I think there’s a lot in here for our community to like. Let’s debate this large selection of options, identify the strongest options, and work together to implement them.
“1. The first part of this Bill establishes a set of national wellbeing goals, formulated by the Secretary of State and confirmed via a public consultation. It places a duty on public bodies and government departments to set objectives in line with these goals, whilst demonstrating certain ‘ways of working’; these are a consideration for the long-term, prevention, planning for risk, collaboration, integration and involvement. Decisions are to be accompanied by future generations impact assessments to ensure longer-term unintended consequences on national wellbeing are mitigated.
2. The second part of this Bill focuses on improving planning and spending within Government. The Bill establishes a futures and forecasting report which assesses the risks and trends, for at least the forthcoming 25 years, and lays out detailed plans on mitigating these risks; the Bill makes provision that when doing so, the views of various relevant groups must be accounted for, including the UK and UN Climate Change Committees and the views of 11-25 year olds on wellbeing. This is to improve the United Kingdom’s preparedness for existential risk. Currently, the Cabinet Office’s National Risk Register only accounts for two years into the future. The Bill also requires departments to categorise their spending into preventative tiers to encourage public bodies to think about investing more money in the short-term to make savings in the long-term, encouraging a pivot towards prevention rather than immediate relief.
3. To improve transparency and accountability within Government, the Bill allocates powers to the head of the National Audit Office to conduct examinations on public bodies in order to assess whether a body has acted in accordance with its wellbeing duties. The Bill extends the Office for Budget Responsibility’s responsibilities to examine the extent to which progress is being made towards the national indicators and subsequent milestones. This, combined with the futures and forecasting report, is used to produce advice to the Treasury to ensure long-term fiscal risks are mitigated. A Joint Select Committee on Future Generations is also established by the Bill to ensure any relevant incoming legislation can be reviewed and amendments suggested. The Bill makes provision for there to be a minister in each Government department in charge of safeguarding future generations’ interests. Their role is to promote the wellbeing goals when formulating policy and, through observing how the Bill is applied within departments, they can also feed back into how the national indicators should be adapted (after consulting with the Joint Committee and the Commission). A Future Generations Commission is to be established, consisting of an expert from each country of the United Kingdom and a young person from each devolved country to improve understanding of the future generations principle amongst public bodies and the public.”
Really excellent comment—could be its own post, arguably.
Note quango isn’t an official term. UK government recognises 4 different types of groups:
Non ministerial departments
Agencies and other public bodies
High profile groups
Public corporations
Some examples:
National Audit Office
Office of Budget Reponsibility
Climate Change Committee
Independent Commission for Aid Impact
National Infrastructure Commission
Centre for the Protection of National Infrastructure
Other examples—WOPR (War Operation Plan Response) from WarGames and the Doomsday Machine from Dr. Strangelove.
I think this is a very interesting project, and I hope it produces lots of great new writing.
I’m wondering what you think the likely effects (if any) might be on the EA Forum?
I will check out neXt, thanks. I like the idea of reboots, very Edge Of Tomorrow.