I guess “narrow target” is just an underspecified part of your argument then, because I don’t know what it’s meant to capture if not “in most plausible scenarios, AI doesn’t follow the same set of rules as humans”.
Can you outline the case for thinking that “in most plausible scenarios, AI doesn’t follow the same set of rules as humans”? To clarify, by “same set of rules” here I’m imagining basic legal rules: do not murder, do not steal etc. I’m not making a claim that specific legal statutes will persist over time.
It seems to me both that:
To the extent that AIs are our descendants, they should inherit our legal system, legal principles, and legal concepts, similar to how e.g. the United States inherited legal principles from the United Kingdom. We should certainly expect our legal system to change over time as our institutions adapt to technological change. But, absent a compelling reason otherwise, it seems wrong to think that “do not murder a human” will go out the window in “most plausible scenarios”.
Our basic legal rules seem pretty natural, rather than being highly contingent. It’s easy to imagine plenty of alien cultures stumbling upon the idea of property rights, and implementing the rule “do not steal from another legal person”.
My point is that AI could plausibly have rules for interacting with other “persons”, and those rules could look much like ours, but that we will not be “persons” under their code. Consider how “do not murder” has never applied to animals.
If AIs treat us like we treat animals then the fact that they have “values” will not be very helpful to us.
I think AIs will be trained on our data, and will be integrated into our culture, having been deliberately designed for the purpose of filling human-shaped holes in our economy, to automate labor. This means they’ll probably inherit our social concepts, in addition to most other concepts we have about the physical world. This situation seems disanalogous to the way humans interact with animals in many ways. Animals can’t even speak language.
Anyway, even the framing you have given seems like a partial concession towards my original point. A rejection of premise 4 is not equivalent to the idea that AIs will automatically follow our legal norms. Instead, it was about whether “human values” are an extremely narrow target, in the sense of being a natural vs. contingent set of values that are very hard to replicate in other circumstances.
If the way AIs relate to human values is similar to how humans relate to animals, then I’ll point out that many existing humans already find the idea of caring about animals to be quite natural, even if most ultimately decide not to take the idea very far. Compare the concept of “caring about animals” to “caring about paperclip maximization”. In the first instance, we have robust examples of people actually doing that, but hardly any examples of people in the second instance. This is after all because caring about paperclip maximization is an unnatural and arbitrary thing to care about relative to how most people conceptualize the world.
Again, I’m not saying AIs will necessarily care about human values. That was never the claim. The entire question was about whether human values are an “extremely narrow target”. And I think, within this context, given the second interpretation of human values in my original comment, the original thesis seems to have held up fine.
I guess “narrow target” is just an underspecified part of your argument then, because I don’t know what it’s meant to capture if not “in most plausible scenarios, AI doesn’t follow the same set of rules as humans”.
Can you outline the case for thinking that “in most plausible scenarios, AI doesn’t follow the same set of rules as humans”? To clarify, by “same set of rules” here I’m imagining basic legal rules: do not murder, do not steal etc. I’m not making a claim that specific legal statutes will persist over time.
It seems to me both that:
To the extent that AIs are our descendants, they should inherit our legal system, legal principles, and legal concepts, similar to how e.g. the United States inherited legal principles from the United Kingdom. We should certainly expect our legal system to change over time as our institutions adapt to technological change. But, absent a compelling reason otherwise, it seems wrong to think that “do not murder a human” will go out the window in “most plausible scenarios”.
Our basic legal rules seem pretty natural, rather than being highly contingent. It’s easy to imagine plenty of alien cultures stumbling upon the idea of property rights, and implementing the rule “do not steal from another legal person”.
My point is that AI could plausibly have rules for interacting with other “persons”, and those rules could look much like ours, but that we will not be “persons” under their code. Consider how “do not murder” has never applied to animals.
If AIs treat us like we treat animals then the fact that they have “values” will not be very helpful to us.
I think AIs will be trained on our data, and will be integrated into our culture, having been deliberately designed for the purpose of filling human-shaped holes in our economy, to automate labor. This means they’ll probably inherit our social concepts, in addition to most other concepts we have about the physical world. This situation seems disanalogous to the way humans interact with animals in many ways. Animals can’t even speak language.
Anyway, even the framing you have given seems like a partial concession towards my original point. A rejection of premise 4 is not equivalent to the idea that AIs will automatically follow our legal norms. Instead, it was about whether “human values” are an extremely narrow target, in the sense of being a natural vs. contingent set of values that are very hard to replicate in other circumstances.
If the way AIs relate to human values is similar to how humans relate to animals, then I’ll point out that many existing humans already find the idea of caring about animals to be quite natural, even if most ultimately decide not to take the idea very far. Compare the concept of “caring about animals” to “caring about paperclip maximization”. In the first instance, we have robust examples of people actually doing that, but hardly any examples of people in the second instance. This is after all because caring about paperclip maximization is an unnatural and arbitrary thing to care about relative to how most people conceptualize the world.
Again, I’m not saying AIs will necessarily care about human values. That was never the claim. The entire question was about whether human values are an “extremely narrow target”. And I think, within this context, given the second interpretation of human values in my original comment, the original thesis seems to have held up fine.