”I don’t have a good inside view on timelines, but when EY says our probability of survival is ~0% this seems like an extraordinary claim that doesn’t seem to be very well supported or argued for, and something I intuitively want to reject outright, but don’t have the object level expertise to meaningfully do so. I don’t know the extent to which EY’s views are representative or highly influential in current AI safety efforts, and I can imagine a world where there’s too much deferring going on. It seems like some within the community have similar thoughts.”
EY’s view of doom being basically certain are fairly marginal. They definitely are part of the conversation, and he certainly is not the only person who holds them. But most people who are actively working on AI safety see the odds of survival as much higher than roughly 0% -- and I think most people see the P(doom) as actually much lower than 80%.
The key motivating argument for AI safety being important, even if you think that EY’s model of the world might be false (though it also might be true) is that while it is easy to come up with plausible reasons to think that P(doom) is much less than 1, it is very hard to dismiss enough of the arguments for it to get p(doom) close to zero.
Yes, I think it’s good that there is basically consensus here on AGI doom being a serious problem; the argument seems to be one of degree. Even OP says p(AGI doom by 2070) ~ 10%.
Object level point:
”I don’t have a good inside view on timelines, but when EY says our probability of survival is ~0% this seems like an extraordinary claim that doesn’t seem to be very well supported or argued for, and something I intuitively want to reject outright, but don’t have the object level expertise to meaningfully do so. I don’t know the extent to which EY’s views are representative or highly influential in current AI safety efforts, and I can imagine a world where there’s too much deferring going on. It seems like some within the community have similar thoughts.”
EY’s view of doom being basically certain are fairly marginal. They definitely are part of the conversation, and he certainly is not the only person who holds them. But most people who are actively working on AI safety see the odds of survival as much higher than roughly 0% -- and I think most people see the P(doom) as actually much lower than 80%.
The key motivating argument for AI safety being important, even if you think that EY’s model of the world might be false (though it also might be true) is that while it is easy to come up with plausible reasons to think that P(doom) is much less than 1, it is very hard to dismiss enough of the arguments for it to get p(doom) close to zero.
Yes, I think it’s good that there is basically consensus here on AGI doom being a serious problem; the argument seems to be one of degree. Even OP says p(AGI doom by 2070) ~ 10%.