I think Carl Shulman makes some persuasive criticisms of this research here :
My main issue with the paper is that it treats existential risk policy as the result of a global collective utility-maximizing decision based on people’s tradeoffs between consumption and danger. But that is assuming away approximately all of the problem.
If we extend that framework to determine how much society would spend on detonating nuclear bombs in war, the amount would be zero and there would be no nuclear arsenals. The world would have undertaken adequate investments in surveillance, PPE, research, and other capacities in response to data about previous coronaviruses such as SARS to stop COVID-19 in its tracks. Renewable energy research funding would be vastly higher than it is today, as would AI technical safety. As advanced AI developments brought AI catstrophic risks closer, there would be no competitive pressures to take risks with global externalities in development either by firms or nation-states.
Externalities massively reduce the returns to risk reduction, with even the largest nation-states being only a small fraction of the world, individual politicians much more concerned with their term of office and individual careers than national-level outcomes, and individual voters and donors constituting only a minute share of the affected parties. And conflict and bargaining problems are entirely responsible for war and military spending, central to the failure to overcome externalities with global climate policy, and core to the threat of AI accident catastrophe.
If those things were solved, and the risk-reward tradeoffs well understood, then we’re quite clearly in a world where we can have very low existential risk and high consumption. But if they’re not solved, the level of consumption is not key: spending on war and dangerous tech that risks global catastrophe can be motivated by the fear of competitive disadvantage/local catastrophe (e.g. being conquered) no matter how high consumption levels are.
I agree with Carl; I feel like other commenters are taking this research as a strong update, as opposed to a simple model which I’m glad someone’s worked through the details of but which we probably shouldn’t use to influence our beliefs very much.
I think Carl Shulman makes some persuasive criticisms of this research here :
I agree with Carl; I feel like other commenters are taking this research as a strong update, as opposed to a simple model which I’m glad someone’s worked through the details of but which we probably shouldn’t use to influence our beliefs very much.