We’ve just released a paper with the above title where we argue that AI safety discussions warrant much greater attention to suffering caused by misused or misaligned AI, as well as from an entire category of overlooked scenarios: uncontrolled AI aligned with values that don’t prioritize suffering prevention. We write, “Technically aligning powerful AI systems with our current, flawed societal values and practices, heterogeneous as they are, would not amount to genuine moral alignment. It would risk preserving and amplifying our present moral blind spots and even locking them in as durable features of the future.” We include the abstract and table below.
Abstract
Large-scale intense suffering resulting from the deployment of AI systems represents a moral blind spot that AI governance has largely overlooked. The underlying issue is not primarily technical but one of values: a failure of societal norms and practices to prioritize suffering prevention. We highlight three categories of AI-driven suffering risks: 1. Human misuse of AI; 2. Uncontrolled misaligned AI; and, in particular, 3. Uncontrolled AI aligned with flawed values. We urge policymakers and developers to: 1. Prioritize suffering prevention in AI governance and development; 2. Assess AI systems’ impacts on suffering; 3. Ensure responsible human control over AI systems; and 4. Pursue international cooperation centered on compassionate values.
Table
Risk category
Key scenarios of AI-driven suffering risks
1. Human misuse of AI
Governments’ pervasive surveillance, repression and punishment
The Moral Blind Spot in AI Governance: Risks of Large-Scale Suffering
We’ve just released a paper with the above title where we argue that AI safety discussions warrant much greater attention to suffering caused by misused or misaligned AI, as well as from an entire category of overlooked scenarios: uncontrolled AI aligned with values that don’t prioritize suffering prevention. We write, “Technically aligning powerful AI systems with our current, flawed societal values and practices, heterogeneous as they are, would not amount to genuine moral alignment. It would risk preserving and amplifying our present moral blind spots and even locking them in as durable features of the future.” We include the abstract and table below.
Abstract
Large-scale intense suffering resulting from the deployment of AI systems represents a moral blind spot that AI governance has largely overlooked. The underlying issue is not primarily technical but one of values: a failure of societal norms and practices to prioritize suffering prevention. We highlight three categories of AI-driven suffering risks: 1. Human misuse of AI; 2. Uncontrolled misaligned AI; and, in particular, 3. Uncontrolled AI aligned with flawed values. We urge policymakers and developers to: 1. Prioritize suffering prevention in AI governance and development; 2. Assess AI systems’ impacts on suffering; 3. Ensure responsible human control over AI systems; and 4. Pursue international cooperation centered on compassionate values.
Table
Risk category
Key scenarios of AI-driven suffering risks
Governments’ pervasive surveillance, repression and punishment
AI-enabled autonomous weapons, cyber/biological warfare
Corporations automating labor (unemployment, inequality)
Algorithmic workplace surveillance (eroded autonomy)
Products exploiting vulnerabilities (addiction, mental health)
Optimizing factory farming (animal suffering)
Creating suffering in AI systems as a by-product of other objectives
Individuals/small groups: bioweapons, cyberattacks, disinformation
Exploiting humans as resources
Suffering as side effect of optimization for other goals
Inflicting suffering for compliance/deterrence
Retaliatory behavior
Gradual human disempowerment through consolidated control
Killing humans as an impediment or as material to exploit
“Benevolent” elimination of sentient life
3. Uncontrolled AI aligned with flawed values
Locking in status quo, preventing progress in suffering prevention
Optimizing non-suffering metrics, neglecting and causing suffering
Replicating lives containing severe, chronic suffering
Scaling factory farming and other cruelty to non-human animals
Expanding wild animal suffering
Creating artificial/digital suffering, as a by-product or conflict strategy