A counterpoint I thought of: it seems like if there is no consequential coevolution happening between humans and other mammalian species, then perhaps AI could grow on such fast time scales that humans can’t hope to have meaningful coevolution.
Maybe. But it seems like we have to pick one: either
(1) Powerful AI tries to impose global permanent totalitarian oppression based on its own stable, locked-in values, preferences, and priorities… which would make it static and brittle, and a sitting duck for coevolution by any beings it’s exploiting,
or
(2) Powerful AI tries to impose oppression based on its own nimble, adaptive, changeable values, preferences, and priorities.… which could co-evolve faster than any beings it’s exploiting, but which would mean it’s no longer ‘permanent’ in terms of the goals and nature of its totalitarian oppression.
Thanks for sharing!
A counterpoint I thought of: it seems like if there is no consequential coevolution happening between humans and other mammalian species, then perhaps AI could grow on such fast time scales that humans can’t hope to have meaningful coevolution.
Maybe. But it seems like we have to pick one: either
(1) Powerful AI tries to impose global permanent totalitarian oppression based on its own stable, locked-in values, preferences, and priorities… which would make it static and brittle, and a sitting duck for coevolution by any beings it’s exploiting,
or
(2) Powerful AI tries to impose oppression based on its own nimble, adaptive, changeable values, preferences, and priorities.… which could co-evolve faster than any beings it’s exploiting, but which would mean it’s no longer ‘permanent’ in terms of the goals and nature of its totalitarian oppression.