I think a key consideration here is whether AI disempowerment of humans, where humans are at least as well off as now, counts as X risk (and, as an aside, P(doom)). Since it would be destruction of humanityās long-term potential, I think Bostrom would say that disempowerment is an X risk, but Ord may not.
This is a good point that I hadnāt thought of when I wrote my poll answerāa āgradual disempowermentā risk scenario would probably not kill all sentient animals, and it represents a non-trivial percentage of AI risk.
I realize I didnāt choose a clear position on this in my description, and Iām actually not sure. Iād call a complete, seemingly irreversible collapse of civilization, even with humans remaining on earth (what the outcome of a nuclear war could be, for example), an X-risk even if itās not full-on extinction, but when it comes to lock-in and disempowerment, since humans (and presumably other animals) remain numerous and living, it doesnāt feel like it should be part of the same question. Iād say my question is about X-risks involving death and destruction (or even mass sterilization), rather than a change in who controls the outcome.
I think a key consideration here is whether AI disempowerment of humans, where humans are at least as well off as now, counts as X risk (and, as an aside, P(doom)). Since it would be destruction of humanityās long-term potential, I think Bostrom would say that disempowerment is an X risk, but Ord may not.
This is a good point that I hadnāt thought of when I wrote my poll answerāa āgradual disempowermentā risk scenario would probably not kill all sentient animals, and it represents a non-trivial percentage of AI risk.
I realize I didnāt choose a clear position on this in my description, and Iām actually not sure. Iād call a complete, seemingly irreversible collapse of civilization, even with humans remaining on earth (what the outcome of a nuclear war could be, for example), an X-risk even if itās not full-on extinction, but when it comes to lock-in and disempowerment, since humans (and presumably other animals) remain numerous and living, it doesnāt feel like it should be part of the same question. Iād say my question is about X-risks involving death and destruction (or even mass sterilization), rather than a change in who controls the outcome.