My point is that AI could plausibly have rules for interacting with other “persons”, and those rules could look much like ours, but that we will not be “persons” under their code. Consider how “do not murder” has never applied to animals.
If AIs treat us like we treat animals then the fact that they have “values” will not be very helpful to us.
I think AIs will be trained on our data, and will be integrated into our culture, having been deliberately designed for the purpose of filling human-shaped holes in our economy, to automate labor. This means they’ll probably inherit our social concepts, in addition to most other concepts we have about the physical world. This situation seems disanalogous to the way humans interact with animals in many ways. Animals can’t even speak language.
Anyway, even the framing you have given seems like a partial concession towards my original point. A rejection of premise 4 is not equivalent to the idea that AIs will automatically follow our legal norms. Instead, it was about whether “human values” are an extremely narrow target, in the sense of being a natural vs. contingent set of values that are very hard to replicate in other circumstances.
If the way AIs relate to human values is similar to how humans relate to animals, then I’ll point out that many existing humans already find the idea of caring about animals to be quite natural, even if most ultimately decide not to take the idea very far. Compare the concept of “caring about animals” to “caring about paperclip maximization”. In the first instance, we have robust examples of people actually doing that, but hardly any examples of people in the second instance. This is after all because caring about paperclip maximization is an unnatural and arbitrary thing to care about relative to how most people conceptualize the world.
Again, I’m not saying AIs will necessarily care about human values. That was never the claim. The entire question was about whether human values are an “extremely narrow target”. And I think, within this context, given the second interpretation of human values in my original comment, the original thesis seems to have held up fine.
My point is that AI could plausibly have rules for interacting with other “persons”, and those rules could look much like ours, but that we will not be “persons” under their code. Consider how “do not murder” has never applied to animals.
If AIs treat us like we treat animals then the fact that they have “values” will not be very helpful to us.
I think AIs will be trained on our data, and will be integrated into our culture, having been deliberately designed for the purpose of filling human-shaped holes in our economy, to automate labor. This means they’ll probably inherit our social concepts, in addition to most other concepts we have about the physical world. This situation seems disanalogous to the way humans interact with animals in many ways. Animals can’t even speak language.
Anyway, even the framing you have given seems like a partial concession towards my original point. A rejection of premise 4 is not equivalent to the idea that AIs will automatically follow our legal norms. Instead, it was about whether “human values” are an extremely narrow target, in the sense of being a natural vs. contingent set of values that are very hard to replicate in other circumstances.
If the way AIs relate to human values is similar to how humans relate to animals, then I’ll point out that many existing humans already find the idea of caring about animals to be quite natural, even if most ultimately decide not to take the idea very far. Compare the concept of “caring about animals” to “caring about paperclip maximization”. In the first instance, we have robust examples of people actually doing that, but hardly any examples of people in the second instance. This is after all because caring about paperclip maximization is an unnatural and arbitrary thing to care about relative to how most people conceptualize the world.
Again, I’m not saying AIs will necessarily care about human values. That was never the claim. The entire question was about whether human values are an “extremely narrow target”. And I think, within this context, given the second interpretation of human values in my original comment, the original thesis seems to have held up fine.