This is a link post for Charles I. Jone’s paper: How Much Should We Spend to Reduce A.I.’s Existential Risk?
Two caveats- I am not Professor Jones, and I take no credit for the linked work. Also this post was written with the help of AI.
Summary of the paper
Stanford economist Charles I. Jones uses standard cost-benefit analysis to estimate how much the US should spend on AI safety. His conclusion: between 1-8% of GDP annually, or roughly $290 billion to $1.5 trillion per year for the US alone.
The core logic is straightforward. US policymakers value a statistical life at around $10 million. To avoid a 1% mortality risk, this implies willingness to pay $100,000 per person. If AI poses similar or greater existential risks over the next decade (which many AI researchers believe it does), comparable investment levels are justified.
Critically, these numbers don’t require any concern for future generations. Jones explicitly models a “selfish” scenario that only values currently living people, and still finds massive spending justified.
Why I think this paper matters
I think this paper helps gives a perspective of how underfunded AI safety remains, despite fairly rapid growth in funding over the past decade.
Global AI safety spending in 2024 was estimated at just over $100m. Jones’s analysis suggests the US alone should be spending 3,000-15,000 times more on AI safety, even without taking non-US or future lives into account.
I know that the idea that existential risk reduction is underfunded is unlikely to take many EAF readers by surprise. However, I think that this paper is worth highlighting. Mainstream economics is a powerful means of both elucidation and legitimation. As J.S. Keynes said: “Practical men who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.”
Even someone as ‘galaxy brained’ and ‘AGI-pilled’ as Tyler Cowen once reportedly said: “[I] would start listening when the AI risk people published in a top economics journal showing the risk is real”.
Here’s the PDF.
I haven’t read it, but I feel like there’s something missing from the summary here, which is like “how much AI risk reduction you get per dollar”. That has to be modeled somehow, right? What did the author assume for that?
If we step outside the economic model into reality, I think reducing AI x-risk is hard, and as evidence we can look around the field and notice that many people trying to reduce AI x-risk are pointing their fingers at many other people trying to reduce AI x-risk, with the former saying that the latter have been making AI x-risk worse rather than better via their poorly-thought-through interventions.
If some institution or government decided to spend $100B per year on AI x-risk (haha), I would be very concerned that this tsunami of money would wind up net negative, leaving us in a worse situation than if the institution / government had spent $0 instead. But of course it would depend a lot on the decisionmakers and processes etc.
Thanks for sharing this. While I think there are strong reasons to invest heavily in AI safety, I’m concerned this particular cost-benefit framing may not be as compelling as it initially appears.
The paper uses a $10 million value of statistical life (VSL) to justify spending $100,000 per person to avoid a 1% mortality risk. However, if we’re being consistent with cost-effectiveness reasoning, we should note that GiveWell-recommended charities save lives in the developing world for approximately $5,000 each—roughly 2,000 times cheaper per life saved.
By this logic, the same funding directed toward global health interventions would save orders of magnitude more lives with near-certainty, compared to reducing AI x-risk with uncertain probability.
This doesn’t mean AI safety is a bad investment—there are strong arguments based on:
The value of preserving future generations (which Jones notes would increase spending estimates)
Diminishing returns or bottlenecks in scaling proven global health interventions
The categorical importance of preventing existential catastrophe
Portfolio diversification across different types of risk
(Note: comment generated in collaboration with AI)
I completely agree with your comment. However my interpretation of what Professor Jones is trying to do is slightly different from straightforward cause prioritisation in the EA sense.
I think he is trying to frame AI risk reduction in a way that is compelling to policymakers, by focusing on standard benchmark values (Value of a Statistical Life), and limiting his analysis in space (only ‘valuing’ lives of American citizens) and time (only the next 20 years). This puts the report in line with standard government Cost Benefit Analyses, which may make it more convincing for those who have access to policy levers.