Executive summary: While individual “doomsday actors” pose some risk, the far greater threat to humanity comes from powerful institutions with concentrated resources and misaligned incentives—a problem that policymakers acknowledge but struggle to address through existing governance structures.
Key points:
Torres (2018a, 2018b) identifies six categories of actors who might destroy humanity if motivated to do so: apocalyptic terrorists, misguided moral actors (like negative utilitarians), radical ecoterrorists, idiosyncratic lone actors, value-misaligned machine superintelligence, and belligerent extraterrestrials.
Kallenborn & Ackerman (2023) find that while various groups have motivation to destroy the world, no terrorist group currently possesses both the will and capability to do so, and focusing on resilience and addressing societal vulnerabilities is the best approach to lower risk.
Meggitt (2020) and Juergensmeyer (2003) show that “apocalyptic terrorism” is deceptive as a category—Aum Shinrikyo’s apocalyptic rhetoric served meaning-making rather than expressing genuine omnicidal intent, and apocalypticism in terrorist groups often shifts strategically with group fortunes.
Sandberg & Nelson (2020) model that successful bioterrorism requires overcoming multiple steps (insights, implementation, evasion, scaling), each requiring resources, making it more likely that powerful institutional actors rather than numerous weak actors would succeed.
Kemp (2021) argues that existential risk is mainly driven by big institutional actors like the military-industrial complex, Big Tech, and the fossil fuel industry, who possess the resources to build nuclear weapons, develop AGI, or shape climate outcomes—with profits privatized and downsides public.
Nathan & Hyams (2022) interviewed policymakers and found they recognize that current solutions are lacking and governance is reactive, but face structural constraints from short-term political pressure and the challenge of regulating institutional actors whose influence spans national boundaries.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: While individual “doomsday actors” pose some risk, the far greater threat to humanity comes from powerful institutions with concentrated resources and misaligned incentives—a problem that policymakers acknowledge but struggle to address through existing governance structures.
Key points:
Torres (2018a, 2018b) identifies six categories of actors who might destroy humanity if motivated to do so: apocalyptic terrorists, misguided moral actors (like negative utilitarians), radical ecoterrorists, idiosyncratic lone actors, value-misaligned machine superintelligence, and belligerent extraterrestrials.
Kallenborn & Ackerman (2023) find that while various groups have motivation to destroy the world, no terrorist group currently possesses both the will and capability to do so, and focusing on resilience and addressing societal vulnerabilities is the best approach to lower risk.
Meggitt (2020) and Juergensmeyer (2003) show that “apocalyptic terrorism” is deceptive as a category—Aum Shinrikyo’s apocalyptic rhetoric served meaning-making rather than expressing genuine omnicidal intent, and apocalypticism in terrorist groups often shifts strategically with group fortunes.
Sandberg & Nelson (2020) model that successful bioterrorism requires overcoming multiple steps (insights, implementation, evasion, scaling), each requiring resources, making it more likely that powerful institutional actors rather than numerous weak actors would succeed.
Kemp (2021) argues that existential risk is mainly driven by big institutional actors like the military-industrial complex, Big Tech, and the fossil fuel industry, who possess the resources to build nuclear weapons, develop AGI, or shape climate outcomes—with profits privatized and downsides public.
Nathan & Hyams (2022) interviewed policymakers and found they recognize that current solutions are lacking and governance is reactive, but face structural constraints from short-term political pressure and the challenge of regulating institutional actors whose influence spans national boundaries.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.