Apologies for being blunt, but the scenario you lay out is full of claims that just seem to completely ignore very facially obvious rebuttals. This would be less bad if you didn’t seem so confident, but as written the perspective strikes me as naive and I would really like an explanation/defense.
Take for example:
Furthermore, since AIs would “get old” too, in the sense of becoming obsolete in the face of new generations of improved AIs, they could also have reason to not expropriate the wealth of vulnerable old agents because they too will be in such a vulnerable position one day, and thus would prefer not to establish a norm of expropriating the type of agent they may one day become.
Setting aside the debatable assumptions about AIs getting “old,” this just seems to completely ignore the literature on collective action problems. If the scenario were such that any one AI agent can expect to get away with defecting (expropriation from older agents) and the norm-breaking requires passing a non-small threshold of such actions, a rational agent will recognize that their defection has minimal impact on what the collective will do, so they may as well do it before others do.
There are multiple other problems in your post, but I don’t think it’s worth the time going through them all. I just felt compelled to comment because I was baffled by the karma on this post, unless it was just people liking it because they agreed with the beginning portion…?
If the scenario were such that any one AI agent can expect to get away with defecting (expropriation from older agents) and the norm-breaking requires passing a non-small threshold of such actions
This isn’t the scenario I intended to describe, since it seems very unlikely that a single agent could get away with mass expropriation. The more likely scenario is that any expropriation that occurs must have been a collective action to begin with, and thus, there’s no problem of coordination that you describe.
This is common in ordinary expropriation in the real world: if you learned that we were one day going to steal all the wealth from people above the age of 90, you’d likely infer that that this decision was decided collectively, rather than being the result of a single lone agent who went and stole all the wealth for themselves.
Your described scenario is instead more similar to ordinary theft, such as robbery. In that case, defection is usually punished by laws against theft, and people generally have non-altruistic reasons to support the enforcement of these laws.
There are multiple other problems in your post, but I don’t think it’s worth the time going through them all. I just felt compelled to comment because I was baffled by the karma on this post
I’m happy for you to critique the rest of the post. As far as I can tell, the only substantive critique you have offered so far seems to contain a misunderstanding of the scenario I described (conflating private lawbreaking from a lone actor with a collective action to expropriate wealth). But it would certainly not be surprising if my arguments had genuine flaws: these are about speculative matters concerning the future.
I don’t find this response to be a compelling defense of what you actually wrote:
since AIs would “get old” too [...] they could also have reason to not expropriate the wealth of vulnerable old agents because they too will be in such a vulnerable position one day
It’s one thing if the argument is “there will be effective enforcement mechanisms which prevent theft,” but the original statement still just seems to imagine that norms will be a non-trivial reason to avoid theft, which seems quite unlikely for a moderately rational agent.
Ultimately, perhaps much of your scenario was trying to convey a different idea from what I see as the straightforward interpretation, but I think it makes it hard for me to productively engage with it, as it feels like engaging with a motte-and-bailey.
the original statement still just seems to imagine that norms will be a non-trivial reason to avoid theft, which seems quite unlikely for a moderately rational agent.
Sorry, I think you’re still conflating two different concepts. I am not claiming:
Social norms will prevent single agents from stealing from others, even in the absence of mechanisms to enforce laws against theft
I am claiming:
Agents will likely not want to establish a collective norm that it’s OK (on a collective level) to expropriate wealth from old, vulnerable individuals. The reason is because most agents will themselves at some point become old, and thus do not want there to be a norm at that time, that would allow their own wealth expropriated from them when they become old.
There are two separate mechanisms at play here. Individual and local instances of theft, like robbery, are typically punished by specific laws. Collective expropriation of groups, while possible in all societies, is usually handled via more decentralized coordination mechanisms, such as social norms.
In other words, if you’re asking me why an AI agent can’t just steal from a human, in my scenario, I’d say that’s because there will (presumably) be laws against theft. But if you’re asking me why the AIs don’t all get up together and steal from the humans collectively, I’d say it’s because they would not want to violate the general norm against expropriation, especially of older, vulnerable groups.
perhaps much of your scenario was trying to convey a different idea from what I see as the straightforward interpretation, but I think it makes it hard for me to productively engage with it, as it feels like engaging with a motte-and-bailey.
For what it’s worth, I asked Claude 3 and GPT-4 to proof-read my essay before I posted, and they both appeared to understand what I said, with almost no misunderstandings, for every single one of my points (from my perspective). I am not bringing this up to claim you are dumb, or anything like that, but I do think it provides evidence that you could probably better understand what I’m saying if you tried to read my words more carefully.
Apologies for being blunt, but the scenario you lay out is full of claims that just seem to completely ignore very facially obvious rebuttals. This would be less bad if you didn’t seem so confident, but as written the perspective strikes me as naive and I would really like an explanation/defense.
Take for example:
Setting aside the debatable assumptions about AIs getting “old,” this just seems to completely ignore the literature on collective action problems. If the scenario were such that any one AI agent can expect to get away with defecting (expropriation from older agents) and the norm-breaking requires passing a non-small threshold of such actions, a rational agent will recognize that their defection has minimal impact on what the collective will do, so they may as well do it before others do.
There are multiple other problems in your post, but I don’t think it’s worth the time going through them all. I just felt compelled to comment because I was baffled by the karma on this post, unless it was just people liking it because they agreed with the beginning portion…?
This isn’t the scenario I intended to describe, since it seems very unlikely that a single agent could get away with mass expropriation. The more likely scenario is that any expropriation that occurs must have been a collective action to begin with, and thus, there’s no problem of coordination that you describe.
This is common in ordinary expropriation in the real world: if you learned that we were one day going to steal all the wealth from people above the age of 90, you’d likely infer that that this decision was decided collectively, rather than being the result of a single lone agent who went and stole all the wealth for themselves.
Your described scenario is instead more similar to ordinary theft, such as robbery. In that case, defection is usually punished by laws against theft, and people generally have non-altruistic reasons to support the enforcement of these laws.
I’m happy for you to critique the rest of the post. As far as I can tell, the only substantive critique you have offered so far seems to contain a misunderstanding of the scenario I described (conflating private lawbreaking from a lone actor with a collective action to expropriate wealth). But it would certainly not be surprising if my arguments had genuine flaws: these are about speculative matters concerning the future.
I don’t find this response to be a compelling defense of what you actually wrote:
It’s one thing if the argument is “there will be effective enforcement mechanisms which prevent theft,” but the original statement still just seems to imagine that norms will be a non-trivial reason to avoid theft, which seems quite unlikely for a moderately rational agent.
Ultimately, perhaps much of your scenario was trying to convey a different idea from what I see as the straightforward interpretation, but I think it makes it hard for me to productively engage with it, as it feels like engaging with a motte-and-bailey.
Sorry, I think you’re still conflating two different concepts. I am not claiming:
Social norms will prevent single agents from stealing from others, even in the absence of mechanisms to enforce laws against theft
I am claiming:
Agents will likely not want to establish a collective norm that it’s OK (on a collective level) to expropriate wealth from old, vulnerable individuals. The reason is because most agents will themselves at some point become old, and thus do not want there to be a norm at that time, that would allow their own wealth expropriated from them when they become old.
There are two separate mechanisms at play here. Individual and local instances of theft, like robbery, are typically punished by specific laws. Collective expropriation of groups, while possible in all societies, is usually handled via more decentralized coordination mechanisms, such as social norms.
In other words, if you’re asking me why an AI agent can’t just steal from a human, in my scenario, I’d say that’s because there will (presumably) be laws against theft. But if you’re asking me why the AIs don’t all get up together and steal from the humans collectively, I’d say it’s because they would not want to violate the general norm against expropriation, especially of older, vulnerable groups.
For what it’s worth, I asked Claude 3 and GPT-4 to proof-read my essay before I posted, and they both appeared to understand what I said, with almost no misunderstandings, for every single one of my points (from my perspective). I am not bringing this up to claim you are dumb, or anything like that, but I do think it provides evidence that you could probably better understand what I’m saying if you tried to read my words more carefully.