Some people are concerned about AI x-risk, and they have P(doom)s in the 5–25% range. I don’t get that. I can’t pass an Ideological Turing Test for someone who sees all these problems, but still expects us to avert extinction with >75% probability. I don’t understand what would lead one to believe that this is what things look like when we’re on track to solving a problem.
P(doom) does not necessarily equal extinction. Paul Christiano had (in 2023) P(AI takeover) at 22%, and P(most humans die from takeover) = 11% (but then other ways of most people dying). But he has much lower probabilities of extinction due to pseudo pico kindness, acausal trade, etc.
P(doom) does not necessarily equal extinction. Paul Christiano had (in 2023) P(AI takeover) at 22%, and P(most humans die from takeover) = 11% (but then other ways of most people dying). But he has much lower probabilities of extinction due to pseudo pico kindness, acausal trade, etc.