Health, technology and catastrophic risk—New Zealand https://adaptresearchwriting.com/blog/
Matt Boyd
Karma: 358
That’s true in theory. But in practice there are only a (small) finite number of items on the list (those that have been formally investigated with a cost-effectiveness analysis). So once those are all funded, then it would make sense to fund more cost-effectiveness analyses to grow the table. We don’t know how ‘worthwhile’ it is to fund most things, so they are not on the table.
Yes, absolutely, and in almost all cases in health the list of desirable things outstrips the funding bar. The ‘league table’ of interventions is longer than the fraction of them that are/can be funded. So in health there is basically never an overhang. The same will be true for EA/GCR/x-risk projects too. So I agree there is likely no ‘overhang’ there either. But it might be that all the possibly worthwhile projects are not yet listed on the ‘league table’ (whether explicitly or implicitly).
Commonly in health economics and prioritisation (eg New Zealand’s Pharmaceutical Management Agency) you calculate the cost-effectiveness (eg cost per QALY) for a given medication, and then rank the desired medications from most to least cost-effective. You then take the budget, and distribute the funds from top until they run out. This is where your rule the line (bar). Nothing below gets funded unless more budget is allocated. If there are items below the bar worth doing then there is a funding constraint, if everything has been funded and there are leftover funds then there is a funding overhang. So it depends on how long the list of cost-effective desirable projects is as to whether there is a shortfall, right amount, or overhang, and that depends on people thinking up projects and adding them to the list. An ‘overhang’ probably stimulates more creativity and thought on potential projects.
Yes, that’s true for an individual. Sorry, I was more meaning the ‘today’ infographic would be for a person born in say 2002, and the 2050 one for someone born in eg 2030. Some confusion because I was replying about ‘medical infographic for x-risks’ generally rather than specifically your point about personal risk.
Book review EA Forum post here
The infographic could perhaps have a ‘today’ and a ‘in 2050’ version, with the bubbles representing the risks being very small for AI ‘today’ compared to eg suicide, or cancer or heart disease, but then becoming much bigger in the 2050 version, illustrating the trajectory. Perhaps the standard medical cause of death bubbles shrink by 2050 illustrating medical progress.
We can quibble over the numbers but I think the point here is basically right, and if not right for AI then probably right for biorisk or some other risks. That point being even if you only look at probabilities in the next few years and only care about people alive today, then these issues appear to be the most salient policy areas. I’ve noted in a recent draft that the velocity of increase in risk (eg from some 0.0001% risk this year, to eg 10% per year in 50 years) results in issues with such probability trajectories being invisible to eg 2-year national risk assessments at present even though area under curve is greater in aggregate than every other risk. But in a sense potentially ‘inevitable’ (for the demonstration risk profiles I dreamed up) over a human lifetime. This then begs the question of how to monitor the trajectory (surely this is one role of national risk assessment, to invest in ‘fire alarms’, but this then requires these risks to be included in the assessment so the monitoring can be prioritized). Persuading policymakers is definitely going to be easier by leveraging decade long actuarial tables than having esoteric discussions about total utilitarianism.
Additionally, in the recent FLI ‘World Building Contest’ the winning entry from Mako Yass made quite a point of the fact that in the world he built the impetus for AI safety and global cooperation on this issue came from the development of very clear and very specific scenario development of how exactly AI could come to kill everyone. This is analogous to Carl Sagan/Turco’s work on nuclear winter in the early 1980s , a specific picture changed minds. We need this for AI.
Thanks Nick, interesting thoughts, great to see this discussion, and appreciated. Is there a timeline for when the initial (21 March deadline) applications will all be decided? As you say, it takes as long as it takes, but has some implications for prioritising tasks (eg deciding whether to commit to less impactful, less-scalable work being offered, and the opportunity costs of this). Is there a list of successful applications?
Rumtin, I think Jack is absolutely right, and our research, in the process of being written up will argue Australia is the most likely successful persisting hub of complexity in a range of nuclear war scenarios. We include a detailed case study of New Zealand (because of familiarity with the issues) but a detailed case study of Australia is begging to be done. There are key issues (mostly focused around trade, energy forms, societal cohesion, infectious disease resilience, awareness of the main risks—not ‘radiation’ like many public think, and for Australia not climate impacts or food either, which is where most nuclear impact research has focused) that could be improved ahead of time, with co-benefits for climate impact, health, resilience to other catastrophes etc. Australia is indeed uniquely positioned here (for a number of reasons that go beyond ‘survival’ and into ‘resilience’ and ‘reboot’ capacity, etc) and policy should include interconnections with NZ policy (sustaining regional trade, security alliance, etc, we’ve identified other potentially surviving/thriving regional partners too) Happy to collaborate on this. I can send you a draft of our paper in maybe 2 weeks.
Updates would be fantastic.
Thanks Rumtin for this, it’s a fantastic resource. One thing I note though is that some of the author listings are out of order (this is actually a problem in Terra’s CSVs too where I think maybe some of the content in your database is imported from). For example, item 70 by ‘Tang’ (who is indeed an author) is actually first-authored by ‘Wagman’ as per the link. I had this problem using Terra, where I kept thinking I was finding papers I’d previously missed, only to discover they were the same paper but with authors in a different order. Maybe at some point a verification/QC process could be implemented (in both these databases, Terra too, to clean them up a little). Great work!
Bunker on island is probably a robust set-up, at least two given volcanic nature of eg Iceland, New Zealand: https://adaptresearchwriting.com/island-refuges/ Synergies/complementarities in island and bunker work should be explored. We’re currently exploring the islands/nuclear winter strand (EA LTFF), and have put in for FTX too.
In a previous project we used the UN FAO food Pocketbook, although I think the way they compile data changed after 2012. We used the ‘kcal production per capita’ metric, from here: https://www.fao.org/publications/card/en/c/a9f447e8-6798-5e82-82b0-a78724bfff03/
You can see what we did in the following two papers:
https://pubmed.ncbi.nlm.nih.gov/33886124/
https://onlinelibrary.wiley.com/doi/abs/10.1111/risa.13398
There are FAO CSVs for more recent years available to download here: https://www.fao.org/faostat/en/#data/FBS
That’s one suggestion.
Did you ever start/do this project, as per your linked G-doc?
Hi, I have quite a lot to say about this, but I’m actually currently writing a research paper on exactly this issue, and will write a full forum post/link-post once it’s completed (ETA June-ish). However, a couple of key observations:
Cost of living is likely to be irrelevant in nuclear aftermath as global finance and economics is in tatters (the value of assets will jump around unpredictably, eg mansions less important than electric vehicles if global oil trade ceases), prices will change dramatically according to scarcity, eg food prices.
Energy independence and food security are probably the most important (>50% combined index value) because without energy food production is slashed to pre-industrial yields, and without food security the risk of unrest is very high.
Latitude and temperature are less important than the impact on specific countries, eg temperature change is important not mean temperature, tropical crops like rice will die in a single frost. Europe could suffer −20 C or −30 C temperature change according to climate models, which makes agriculture impossible. Yet Iceland with vast fish resources could potentially increase food production.
Rainfall could have a massive impact. The tropical monsoons could be very disrupted and are essential for agriculture in many areas.
The could very well be almost no trade taking place in a severe nuclear aftermath as nations struggle internally, or due to fuel shortages (many countries are dependent on oil for agriculture at scale). Without trade many countries are fragile in areas of energy and manufacturing. Many component parts of power generation facilities, electricity & food distribution and communications infrastructure are manufactured in only a few places and within a few months without imports/exports such infrastructure may fail (eg lubricants, spark plugs, transformers, fibre optics, etc). Expect most things to grind to a halt without trade.
There is a lot more that could be said but you’re right that the large South American food producers (Argentina etc) look relatively more promising, as well as the usual suspects NZ & Australia. Though each will have severe problems in an actual nuclear winter and organisation such as food/fuel rationing and distribution from rural to urban areas will be immensely problematic. Not to mention the need for public communication processes to ensure people know there is a plan and survival is possible, again to avoid societal mayhem. Social cohesion, and stability indicators are probably very important.
One problem with composite indices is that very low scores on one dimension can be masked by reasonable scores on others. Countries should be ruled out if they fail on a critical dimension.
Finally, the act of ‘escaping to’ the ‘most promising’ location is not generalisable, and so the ethics of it are questionable. As Kant notes, the test is ‘what if everyone did the same as me, would that undermine the institution in question?’ and in this case it seems like the answer is yes. 8 billion people fleeing to Argentina would defeat the purpose of acting ahead of war to maximise the chances of each particular country. Carrying capacity calculations are important here too. I haven’t even considered HEMP yet, which could very much complicate matters.
The following case study is particularly illuminating of the problems even ‘good’ locations like NZ might suffer: https://www.jstor.org/stable/4313623?refreqid=excelsior%3A166e17f569637767a9caded49a1ced42 contact me if you want the full text.
‘Partitioning’ is another concept that might be useful.
Islands as refuge (basically same idea as the city idea above), this paper specifically mentions pandemic as threat and island as solution (ie risk first approach) and also considers nuclear (and other) winter scenarios too (see the Supplementary material): https://pubmed.ncbi.nlm.nih.gov/33886124/
I note Alexey’s comment here too, broadly agree with his islands/refuge thinking.
The literature on group selection and species selection in biology might prove useful. You seem to be on to it tangentially with the butterfly example.
I enjoyed this. Would seem to do well as an argument for preventing existential risk from Scheffler’s ‘the human project’ point of view, ie the continuation of transgenerational undertakings that we each contribute a tiny piece to, as opposed to the maximizing total utility approach. Persistence of the whole seems to have emergent merit beyond the lives of the individuals.
On the other hand it also made me think of the line Chigurh says in ‘No Country for Old Men’ > “If the rule that you followed brought you to this, of what use was the rule?” Rule = eg not eating meat, being compassionate etc. [note, I believe there IS use in the rules, but the line still haunts me]
Thanks Carla and Luke for a great paper. This is exactly the sort of antagonism that those not so deeply immersed in the xrisk literature can benefit from, because it surveys so much and highlights the dangers of a single core framework. Alternatives to the often esoteric and quasi-religious far-future speculations that seem to drive a lot of xrisk work are not always obvious to decision makers and that gap means that the field can be ignored as ‘far fetched’. Democratisation is a critical component (along with apoliticisation).
I must say that it was a bit of a surprise to me that TUA is seen as the paradigm approach to ERS. I’ve worked in this space for about 5-6 years and never really felt that I was drawn to strong-longtermism or transhumanism, or technological progress. ERS seems like the limiting case of ordinary risk studies to me. I’ve worked in healthcare quality and safety (risk to one person at a time), public health (risk to members of populations) and extinction risk just seems like the important and interesting limit of this. I concur with the calls for grounding in the literature of risk analysis, democracy, and pluralism. In fact in peer reviewed work I’ve previously called for citizen juries and public deliberation and experimental philosophy in this space (here), and for apolitical, aggregative processes (here), as well as calling for better publicly facing national risk (and xrisk) communication and prioritisation tools (under review with Risk Analysis).
Some key points I appreciated or reflected on in your paper were:
The fact that empirical and normative assumptions are often masked by tools and frameworks
The distinction between extinction risk and existential risk.
The questioning of total utilitarianism (I often prefer a maximin approach, also with consideration of important [not necessarily maximising] value obtained from honouring treaties, equity, etc)
I’ve never found the ‘astronomical waste’ claims hold up particularly well under certain resolutions of Fermi’s paradox (basically I doubt the moral and empirical claims of TUA and strong longtermism, and yet I am fully committed to ERS)
The point about equivocating over near-term nuclear war and billion year stagnation
Clarity around Ord’s 1 in 6 (extinction/existential) - I’m guilty of conflating this
I note that failing to mitigate ‘mere’ GCRs could also derail certain xrisk mitigation efforts.
Again, great work. This is a useful and important broad survey/stimulus, not every paper needs to take a single point and dive to its bottom. Well done.
Thanks for these comments Noumero, much appreciated!
Hi Christian, thanks for your thoughts. You’re right to note that islands like Iceland, Indonesia, NZ, etc are also where there’s a lot of volcanic activity. Mike Cassidy and Lara Mani briefly summarize potential ash damage in their post on supervolcanoes here (see the table on effects). Basically there could be severe impacts on agriculture and infrastructure. I think the main lesson is that at least two prepared islands would be good. In different hemispheres. That first line of redundancy is probably the most important (also in case one is a target in nuclear war, eg NZ is probably susceptible to an EMP directed at Australia).