Executive summary: The author argues that AI safety planning is dangerously over-reliant on long chains of conjunctive conditions, and calls for “breadth-first” plans that maintain multiple independent paths to success so that the overall effort survives even when individual assumptions fail.
Key points:
“Depth-first” AI safety plans fail entirely if any single condition in their chain is false, and the author counts at least eight such conditions in Google’s April 2025 safety plan alone.
The author argues that disjunctive conditions (where success requires A or B or C) are preferable to conjunctive ones, because fewer simultaneous assumptions need to hold.
A “breadth-first” plan instead pursues multiple actions X, Y, and Z, each depending on different conditions, so the overall plan can succeed even if two out of three conditions fail.
The author identifies Barnett & Scher’s AI Governance to Avoid Extinction as the broadest published plan, noting it explicitly maps four possible future scenarios and the conditions required for success in each.
The author sees two main benefits to breadth-first planning: identifying which paths to success depend on the fewest conditions, and making it easier to spot the biggest holes in a plan.
The author calls on AI companies to publish breadth-first plans addressing what they will do if a step in their mainline plan fails, and on governments to legislate that companies cover a defined list of possible future scenarios.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The author argues that AI safety planning is dangerously over-reliant on long chains of conjunctive conditions, and calls for “breadth-first” plans that maintain multiple independent paths to success so that the overall effort survives even when individual assumptions fail.
Key points:
“Depth-first” AI safety plans fail entirely if any single condition in their chain is false, and the author counts at least eight such conditions in Google’s April 2025 safety plan alone.
The author argues that disjunctive conditions (where success requires A or B or C) are preferable to conjunctive ones, because fewer simultaneous assumptions need to hold.
A “breadth-first” plan instead pursues multiple actions X, Y, and Z, each depending on different conditions, so the overall plan can succeed even if two out of three conditions fail.
The author identifies Barnett & Scher’s AI Governance to Avoid Extinction as the broadest published plan, noting it explicitly maps four possible future scenarios and the conditions required for success in each.
The author sees two main benefits to breadth-first planning: identifying which paths to success depend on the fewest conditions, and making it easier to spot the biggest holes in a plan.
The author calls on AI companies to publish breadth-first plans addressing what they will do if a step in their mainline plan fails, and on governments to legislate that companies cover a defined list of possible future scenarios.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.