This is interesting! But I don’t think I fully understand why ‘private law’ rights are supposed to be more credible than ‘wellbeing’ rights. Wouldn’t humans also have an incentive to disregard these ‘private law’ rights when it suited them?
For example, if the laws were still created by humans, then if AI systems accumulated massive amounts of wealth, wouldn’t there be a huge incentive to simply pass a law to confiscate it?
They model that, and after, I think, 1661 iterations of the human-AI trade game, the human-AI trade game accumulates enough wealth for humans that it would’ve been self-defeating for the humans to defect like that. I think it’s still a Nash equilibrium but one where the humans give up perfectly good gains from trade. (Plus blockchain tech can make it hard to confiscate property.)
This is interesting! But I don’t think I fully understand why ‘private law’ rights are supposed to be more credible than ‘wellbeing’ rights. Wouldn’t humans also have an incentive to disregard these ‘private law’ rights when it suited them?
For example, if the laws were still created by humans, then if AI systems accumulated massive amounts of wealth, wouldn’t there be a huge incentive to simply pass a law to confiscate it?
They model that, and after, I think, 1661 iterations of the human-AI trade game, the human-AI trade game accumulates enough wealth for humans that it would’ve been self-defeating for the humans to defect like that. I think it’s still a Nash equilibrium but one where the humans give up perfectly good gains from trade. (Plus blockchain tech can make it hard to confiscate property.)