Executive summary: An exploratory, somewhat urgent argument that AI safety is losing ground to “safety washing” (performative, low-cost nods to safety) and entrenched incentives; the author contends incremental coalition-building is unlikely to suffice and urges preparing for moments of sharp Overton-window shift—most plausibly an AI disaster (secondarily mass unemployment)—by building plans, capacity, and agility to seize those openings.
Key points:
Diagnosis: The Paris AI Action conference and recent policy/lab developments exemplify “safety washing,” where institutions foreground trust/brand or narrow risks while sidelining catastrophic-risk mitigation; overall, AI safety has suffered setbacks across governments, labs, and legislation.
Incentives & rhetoric: Many actors (politicians, labs, startups, open-source, media, etc.) find AI-safety claims inconvenient and engage in motivated reasoning; common tactics include ad hominem, misdirection, overconfident assertions, and naïve skepticism—effective because they offer low-information audiences a “fig leaf.”
Why incrementalism struggles: The community is “outgunned,” timelines may be short, and the offense–defense balance could favor attackers; large coalitions are slow and compromise-prone, while eked-out incremental wins are likely insufficient for the level/speed of risk.
Strategic proposal: Shift primary planning toward leveraging rare moments of dramatic advantage—especially an AI disaster (and, secondarily, salient unemployment shocks)—that could force decisive policy; success depends on pre-planning, capacity, bravery (saying unfashionable truths), and rapid execution.
Contingencies & uncertainties: ChatGPT-style “wake-ups” may no longer move opinion; capabilities could plateau (in which case pivot back to growth/credibility/movement-building); national-security framings may or may not resonate with current U.S. leadership.
Implication for the AI-safety community: Maintain epistemic standards but get more politically realistic; invest now in preparedness for window-shifting events (e.g., agile governance mechanisms, AISI-like capacity) while downgrading expectations for near-term broad-coalition wins.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: An exploratory, somewhat urgent argument that AI safety is losing ground to “safety washing” (performative, low-cost nods to safety) and entrenched incentives; the author contends incremental coalition-building is unlikely to suffice and urges preparing for moments of sharp Overton-window shift—most plausibly an AI disaster (secondarily mass unemployment)—by building plans, capacity, and agility to seize those openings.
Key points:
Diagnosis: The Paris AI Action conference and recent policy/lab developments exemplify “safety washing,” where institutions foreground trust/brand or narrow risks while sidelining catastrophic-risk mitigation; overall, AI safety has suffered setbacks across governments, labs, and legislation.
Incentives & rhetoric: Many actors (politicians, labs, startups, open-source, media, etc.) find AI-safety claims inconvenient and engage in motivated reasoning; common tactics include ad hominem, misdirection, overconfident assertions, and naïve skepticism—effective because they offer low-information audiences a “fig leaf.”
Why incrementalism struggles: The community is “outgunned,” timelines may be short, and the offense–defense balance could favor attackers; large coalitions are slow and compromise-prone, while eked-out incremental wins are likely insufficient for the level/speed of risk.
Strategic proposal: Shift primary planning toward leveraging rare moments of dramatic advantage—especially an AI disaster (and, secondarily, salient unemployment shocks)—that could force decisive policy; success depends on pre-planning, capacity, bravery (saying unfashionable truths), and rapid execution.
Contingencies & uncertainties: ChatGPT-style “wake-ups” may no longer move opinion; capabilities could plateau (in which case pivot back to growth/credibility/movement-building); national-security framings may or may not resonate with current U.S. leadership.
Implication for the AI-safety community: Maintain epistemic standards but get more politically realistic; invest now in preparedness for window-shifting events (e.g., agile governance mechanisms, AISI-like capacity) while downgrading expectations for near-term broad-coalition wins.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.