Executive summary: Superforecasters’ aggregated probabilities differ from the author’s original report on the premises of power-seeking AI as an existential risk, with higher probabilities on some premises and lower probabilities on others.
Key points:
Superforecasters rated premises on AI timelines and incentives higher but alignment difficulty and impact of failures lower than the original report.
The overall probability of existential catastrophe by 2070 was 1% versus 5% originally.
The author has not substantially updated given unpersuasive arguments and uncertainty about deferring to group probabilities.
Engaging with reasoned disagreements is an open question in forecasting methodology.
The author sees this as similar to determining how much to update based on disagreements from markets and economists.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: Superforecasters’ aggregated probabilities differ from the author’s original report on the premises of power-seeking AI as an existential risk, with higher probabilities on some premises and lower probabilities on others.
Key points:
Superforecasters rated premises on AI timelines and incentives higher but alignment difficulty and impact of failures lower than the original report.
The overall probability of existential catastrophe by 2070 was 1% versus 5% originally.
The author has not substantially updated given unpersuasive arguments and uncertainty about deferring to group probabilities.
Engaging with reasoned disagreements is an open question in forecasting methodology.
The author sees this as similar to determining how much to update based on disagreements from markets and economists.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
I think the “alignment difficulty” premise was given higher probability by superforecasters, not lower probability.