Health, technology and catastrophic risk—New Zealand https://adaptresearchwriting.com/blog/
Matt Boyd
Karma: 356
I generally think that all these kinds of cost-effectiveness analyses around x-risk are wildly speculative and susceptible to small changes in assumptions. There is literally no evidence that the $250b would change bio-x-risk by 1% rather than, say, 0.1% or 10%, or even 50%, depending on how it was targeted and what developments it led to. On the other hand if you do successfully reduce the x-risk by, say, 1%, then you most likely also reduce the risk/consequences of all kinds of other non-existential bio-risks, again depending on the actual investment/discoveries/developments, so the benefit of all the ‘ordinary’ cases must be factored in. I think that the most compelling argument for investing in x-risk prevention without consideration of future generations, is simply to calculate the deaths in expectation (eg using Ord’s probabilities if you are comfortable with them) and to rank risks accordingly. It turns out that at 10% this century, AI risks 8 million lives per annum (obviously less than that early century, perhaps greater late century) and bio-risk is 2.7 million lives per annum in expectation (ie 8 billion x 0.0333 x 0.01). This can be compared to ALL natural disasters which Our World in Data reports kill ~60,000 people per annum. So there is an argument that we should focus on x-risk to at least some degree purely on expected consequences. I think its basically impossible to get robust cost-effectiveness estimates for this kind of work, and most of the estimates I’ve seen appear implausibly cost-effective. Things never go as well as you though they would in risk mitigation activities.
Hi Christian, thanks for your thoughts. You’re right to note that islands like Iceland, Indonesia, NZ, etc are also where there’s a lot of volcanic activity. Mike Cassidy and Lara Mani briefly summarize potential ash damage in their post on supervolcanoes here (see the table on effects). Basically there could be severe impacts on agriculture and infrastructure. I think the main lesson is that at least two prepared islands would be good. In different hemispheres. That first line of redundancy is probably the most important (also in case one is a target in nuclear war, eg NZ is probably susceptible to an EMP directed at Australia).
Islands, nuclear winter, and trade disruption as a human existential risk factor
That’s true in theory. But in practice there are only a (small) finite number of items on the list (those that have been formally investigated with a cost-effectiveness analysis). So once those are all funded, then it would make sense to fund more cost-effectiveness analyses to grow the table. We don’t know how ‘worthwhile’ it is to fund most things, so they are not on the table.
Yes, absolutely, and in almost all cases in health the list of desirable things outstrips the funding bar. The ‘league table’ of interventions is longer than the fraction of them that are/can be funded. So in health there is basically never an overhang. The same will be true for EA/GCR/x-risk projects too. So I agree there is likely no ‘overhang’ there either. But it might be that all the possibly worthwhile projects are not yet listed on the ‘league table’ (whether explicitly or implicitly).
Commonly in health economics and prioritisation (eg New Zealand’s Pharmaceutical Management Agency) you calculate the cost-effectiveness (eg cost per QALY) for a given medication, and then rank the desired medications from most to least cost-effective. You then take the budget, and distribute the funds from top until they run out. This is where your rule the line (bar). Nothing below gets funded unless more budget is allocated. If there are items below the bar worth doing then there is a funding constraint, if everything has been funded and there are leftover funds then there is a funding overhang. So it depends on how long the list of cost-effective desirable projects is as to whether there is a shortfall, right amount, or overhang, and that depends on people thinking up projects and adding them to the list. An ‘overhang’ probably stimulates more creativity and thought on potential projects.
Yes, that’s true for an individual. Sorry, I was more meaning the ‘today’ infographic would be for a person born in say 2002, and the 2050 one for someone born in eg 2030. Some confusion because I was replying about ‘medical infographic for x-risks’ generally rather than specifically your point about personal risk.
Book review EA Forum post here
The infographic could perhaps have a ‘today’ and a ‘in 2050’ version, with the bubbles representing the risks being very small for AI ‘today’ compared to eg suicide, or cancer or heart disease, but then becoming much bigger in the 2050 version, illustrating the trajectory. Perhaps the standard medical cause of death bubbles shrink by 2050 illustrating medical progress.
We can quibble over the numbers but I think the point here is basically right, and if not right for AI then probably right for biorisk or some other risks. That point being even if you only look at probabilities in the next few years and only care about people alive today, then these issues appear to be the most salient policy areas. I’ve noted in a recent draft that the velocity of increase in risk (eg from some 0.0001% risk this year, to eg 10% per year in 50 years) results in issues with such probability trajectories being invisible to eg 2-year national risk assessments at present even though area under curve is greater in aggregate than every other risk. But in a sense potentially ‘inevitable’ (for the demonstration risk profiles I dreamed up) over a human lifetime. This then begs the question of how to monitor the trajectory (surely this is one role of national risk assessment, to invest in ‘fire alarms’, but this then requires these risks to be included in the assessment so the monitoring can be prioritized). Persuading policymakers is definitely going to be easier by leveraging decade long actuarial tables than having esoteric discussions about total utilitarianism.
Additionally, in the recent FLI ‘World Building Contest’ the winning entry from Mako Yass made quite a point of the fact that in the world he built the impetus for AI safety and global cooperation on this issue came from the development of very clear and very specific scenario development of how exactly AI could come to kill everyone. This is analogous to Carl Sagan/Turco’s work on nuclear winter in the early 1980s , a specific picture changed minds. We need this for AI.
Thanks Nick, interesting thoughts, great to see this discussion, and appreciated. Is there a timeline for when the initial (21 March deadline) applications will all be decided? As you say, it takes as long as it takes, but has some implications for prioritising tasks (eg deciding whether to commit to less impactful, less-scalable work being offered, and the opportunity costs of this). Is there a list of successful applications?
Rumtin, I think Jack is absolutely right, and our research, in the process of being written up will argue Australia is the most likely successful persisting hub of complexity in a range of nuclear war scenarios. We include a detailed case study of New Zealand (because of familiarity with the issues) but a detailed case study of Australia is begging to be done. There are key issues (mostly focused around trade, energy forms, societal cohesion, infectious disease resilience, awareness of the main risks—not ‘radiation’ like many public think, and for Australia not climate impacts or food either, which is where most nuclear impact research has focused) that could be improved ahead of time, with co-benefits for climate impact, health, resilience to other catastrophes etc. Australia is indeed uniquely positioned here (for a number of reasons that go beyond ‘survival’ and into ‘resilience’ and ‘reboot’ capacity, etc) and policy should include interconnections with NZ policy (sustaining regional trade, security alliance, etc, we’ve identified other potentially surviving/thriving regional partners too) Happy to collaborate on this. I can send you a draft of our paper in maybe 2 weeks.
Updates would be fantastic.
Thanks Rumtin for this, it’s a fantastic resource. One thing I note though is that some of the author listings are out of order (this is actually a problem in Terra’s CSVs too where I think maybe some of the content in your database is imported from). For example, item 70 by ‘Tang’ (who is indeed an author) is actually first-authored by ‘Wagman’ as per the link. I had this problem using Terra, where I kept thinking I was finding papers I’d previously missed, only to discover they were the same paper but with authors in a different order. Maybe at some point a verification/QC process could be implemented (in both these databases, Terra too, to clean them up a little). Great work!
Bunker on island is probably a robust set-up, at least two given volcanic nature of eg Iceland, New Zealand: https://adaptresearchwriting.com/island-refuges/ Synergies/complementarities in island and bunker work should be explored. We’re currently exploring the islands/nuclear winter strand (EA LTFF), and have put in for FTX too.
In a previous project we used the UN FAO food Pocketbook, although I think the way they compile data changed after 2012. We used the ‘kcal production per capita’ metric, from here: https://www.fao.org/publications/card/en/c/a9f447e8-6798-5e82-82b0-a78724bfff03/
You can see what we did in the following two papers:
https://pubmed.ncbi.nlm.nih.gov/33886124/
https://onlinelibrary.wiley.com/doi/abs/10.1111/risa.13398
There are FAO CSVs for more recent years available to download here: https://www.fao.org/faostat/en/#data/FBS
That’s one suggestion.
Did you ever start/do this project, as per your linked G-doc?
Hi, I have quite a lot to say about this, but I’m actually currently writing a research paper on exactly this issue, and will write a full forum post/link-post once it’s completed (ETA June-ish). However, a couple of key observations:
Cost of living is likely to be irrelevant in nuclear aftermath as global finance and economics is in tatters (the value of assets will jump around unpredictably, eg mansions less important than electric vehicles if global oil trade ceases), prices will change dramatically according to scarcity, eg food prices.
Energy independence and food security are probably the most important (>50% combined index value) because without energy food production is slashed to pre-industrial yields, and without food security the risk of unrest is very high.
Latitude and temperature are less important than the impact on specific countries, eg temperature change is important not mean temperature, tropical crops like rice will die in a single frost. Europe could suffer −20 C or −30 C temperature change according to climate models, which makes agriculture impossible. Yet Iceland with vast fish resources could potentially increase food production.
Rainfall could have a massive impact. The tropical monsoons could be very disrupted and are essential for agriculture in many areas.
The could very well be almost no trade taking place in a severe nuclear aftermath as nations struggle internally, or due to fuel shortages (many countries are dependent on oil for agriculture at scale). Without trade many countries are fragile in areas of energy and manufacturing. Many component parts of power generation facilities, electricity & food distribution and communications infrastructure are manufactured in only a few places and within a few months without imports/exports such infrastructure may fail (eg lubricants, spark plugs, transformers, fibre optics, etc). Expect most things to grind to a halt without trade.
There is a lot more that could be said but you’re right that the large South American food producers (Argentina etc) look relatively more promising, as well as the usual suspects NZ & Australia. Though each will have severe problems in an actual nuclear winter and organisation such as food/fuel rationing and distribution from rural to urban areas will be immensely problematic. Not to mention the need for public communication processes to ensure people know there is a plan and survival is possible, again to avoid societal mayhem. Social cohesion, and stability indicators are probably very important.
One problem with composite indices is that very low scores on one dimension can be masked by reasonable scores on others. Countries should be ruled out if they fail on a critical dimension.
Finally, the act of ‘escaping to’ the ‘most promising’ location is not generalisable, and so the ethics of it are questionable. As Kant notes, the test is ‘what if everyone did the same as me, would that undermine the institution in question?’ and in this case it seems like the answer is yes. 8 billion people fleeing to Argentina would defeat the purpose of acting ahead of war to maximise the chances of each particular country. Carrying capacity calculations are important here too. I haven’t even considered HEMP yet, which could very much complicate matters.
The following case study is particularly illuminating of the problems even ‘good’ locations like NZ might suffer: https://www.jstor.org/stable/4313623?refreqid=excelsior%3A166e17f569637767a9caded49a1ced42 contact me if you want the full text.
‘Partitioning’ is another concept that might be useful.
Islands as refuge (basically same idea as the city idea above), this paper specifically mentions pandemic as threat and island as solution (ie risk first approach) and also considers nuclear (and other) winter scenarios too (see the Supplementary material): https://pubmed.ncbi.nlm.nih.gov/33886124/
I note Alexey’s comment here too, broadly agree with his islands/refuge thinking.
The literature on group selection and species selection in biology might prove useful. You seem to be on to it tangentially with the butterfly example.
Hi Ross, here’s the paper that I mentioned in my comment above (this pre-print uses some data from Xia et al 2022 in its preprint form, and their paper has just been published in Nature Food with some slightly updated numbers, so we’ll update our own once the peer review comes back, but the conclusions etc won’t change): https://www.researchsquare.com/article/rs-1927222/v1
We’re now starting a ‘NZ Catastrophe Resilience Project’ to more fully work up the skeleton details that are listed in Supplementary Table S1 of our paper. Engaging with public sector, industry, academia etc. Australia could do exactly the same.
Note that in the Xia paper, NZ’s food availability is vastly underestimated due to quirks of the UNFAO dataset. For an estimate of NZ’s export calories see our paper here: https://www.medrxiv.org/content/10.1101/2022.05.13.22275065v1
And we’ve posted here on the Forum about all this here: https://forum.effectivealtruism.org/posts/7arEfmLBX2donjJyn/islands-nuclear-winter-and-trade-disruption-as-a-human