I sympathise with/âagree with many of your points here (and in general regard AI x-risk), but something about this recent sequence of quick-takes isnât landing with me in the way some of your other work has. Iâll try and articulate why in some cases, though I apologies if I misread or misunderstand you.
On this post, these two presises/âstatements raised an eyebrow:
3. Instead, AI agents will compromise, trade, and act within a system of laws indefinitely, in order to achieve their objectives, similar to what humans do now
4. Because this system of laws will descend from our current institutions and legal tradition, it is likely that humans will keep substantial legal rights, potentially retaining lots of wealth from our capital investments and property, even if we become relatively powerless compared to other AI agents in the system
To me, this is just as unsupported as people who are incredibly certain that there will be âtreacherous turnâ. I get this a supposition/âalternative hypothesis, but how can you possible hold a premise that a system of laws will persist indefinitely? This sort of reminds me of the Leahy/âBach discussion where Bach just says âitâs going to align itself with us if it wants to if it likes us if it loves usâ.I kinda want more that if weâre going to build these powerful systems, saying âtrust me bro, itâll follow our laws and norms and love us backâ doesnât sound very convincing to me. (For clarity, I donât think this is your position or framing, and Iâm not a fan of the classic/âYudkowskian risk position. I want to say I find both perspectives unconvincing)
Secondly, people abide by systems of laws and norms, but we also have many cases of where individuals/âparties/âgroups overturned these norms when they had accumulated enough power and didnât feel the need to abide by the existing regime. This doesnât have to look like the traditional DSA model where humanity gets instantly wiped out, but I donât see why there couldnât be a future where an AI makes move like Sulla using force to overthrow and depower the opposing factions, or the 18 Brumaire.
I sympathise with/âagree with many of your points here (and in general regard AI x-risk), but something about this recent sequence of quick-takes isnât landing with me in the way some of your other work has. Iâll try and articulate why in some cases, though I apologies if I misread or misunderstand you.
On this post, these two presises/âstatements raised an eyebrow:
To me, this is just as unsupported as people who are incredibly certain that there will be âtreacherous turnâ. I get this a supposition/âalternative hypothesis, but how can you possible hold a premise that a system of laws will persist indefinitely? This sort of reminds me of the Leahy/âBach discussion where Bach just says âitâs going to align itself with us if it wants to if it likes us if it loves usâ. I kinda want more that if weâre going to build these powerful systems, saying âtrust me bro, itâll follow our laws and norms and love us backâ doesnât sound very convincing to me. (For clarity, I donât think this is your position or framing, and Iâm not a fan of the classic/âYudkowskian risk position. I want to say I find both perspectives unconvincing)
Secondly, people abide by systems of laws and norms, but we also have many cases of where individuals/âparties/âgroups overturned these norms when they had accumulated enough power and didnât feel the need to abide by the existing regime. This doesnât have to look like the traditional DSA model where humanity gets instantly wiped out, but I donât see why there couldnât be a future where an AI makes move like Sulla using force to overthrow and depower the opposing factions, or the 18 Brumaire.