You should talk to David Pearce. His view of physicalism (phenomenal binding) precludes consciousness in digital minds[1].
But he also goes further and claims that world-ending ASI is impossible for the reason of it requiring, yet lacking, unitary conscious experience (whereas I think that a “blind idiot god” is perfectly capable of destroying everything we value).
You should talk to David Pearce. His view of physicalism (phenomenal binding) precludes consciousness in digital minds[1].
But he also goes further and claims that world-ending ASI is impossible for the reason of it requiring, yet lacking, unitary conscious experience (whereas I think that a “blind idiot god” is perfectly capable of destroying everything we value).