Is everyone willing to accept that “whatever human happens to build the hard takeoff AI gets to be the human the AI is aligned with”? Do AI alignment researchers realize this human may not be them, and may not align with them? Are AI alignment researchers all OK with Vladimir Putin, Kim Jong Un, or Xi Jinping being the alignment target? What about someone like Ted Kaczynski?
If the idea is “we’ll just decide collectively”, then in the most optimistic scenario we can assume (based on our history with democracy) that the alignment target will be something akin to today’s world leaders, none of whom I would be comfortable having an AI aligned with.
If the plan is “we’ll decide collectively, but using a better mechanism than every current existing mechanism” then it feels like there is an implication here that not only can we solve AI alignment but we can also solve human alignment (something humans have been trying and failing to solve for millennia).
Separately, I’m curious why my post got downvoted on quality (not sure if you or someone else). I’m new to this community so perhaps there is some rule I unintentionally broke that I would like to be made aware of.
I’m not necessarily representing this point of view myself, but I think the idea is that any alignment scenario — alignment with any human or group of humans — would be a triumph compared to “doom”.
I do think that in practice if the alignment problem is solved, then yes, whoever gets there first would get to decide. That might not be as bad as you think, though; China is repressive in order to maintain social control, but that repression wouldn’t necessarily be a prerequisite to social control in a super-AGI scenario.
How do we choose which human gets aligned with?
Is everyone willing to accept that “whatever human happens to build the hard takeoff AI gets to be the human the AI is aligned with”? Do AI alignment researchers realize this human may not be them, and may not align with them? Are AI alignment researchers all OK with Vladimir Putin, Kim Jong Un, or Xi Jinping being the alignment target? What about someone like Ted Kaczynski?
If the idea is “we’ll just decide collectively”, then in the most optimistic scenario we can assume (based on our history with democracy) that the alignment target will be something akin to today’s world leaders, none of whom I would be comfortable having an AI aligned with.
If the plan is “we’ll decide collectively, but using a better mechanism than every current existing mechanism” then it feels like there is an implication here that not only can we solve AI alignment but we can also solve human alignment (something humans have been trying and failing to solve for millennia).
Separately, I’m curious why my post got downvoted on quality (not sure if you or someone else). I’m new to this community so perhaps there is some rule I unintentionally broke that I would like to be made aware of.
I did not downvote your post.
I’m not necessarily representing this point of view myself, but I think the idea is that any alignment scenario — alignment with any human or group of humans — would be a triumph compared to “doom”.
I do think that in practice if the alignment problem is solved, then yes, whoever gets there first would get to decide. That might not be as bad as you think, though; China is repressive in order to maintain social control, but that repression wouldn’t necessarily be a prerequisite to social control in a super-AGI scenario.