Sure. In practice there’s the national sovereignty angle though. This just devolves to each party “complies” with the agreement, violating it in various ways. Too much incentive to defect.
The US government just never audits its secret national labs, China just never checks anything, Israel just openly decides they can’t afford to comply at all etc. Everyone claims to be in compliance.
Really depends on how much of a taboo develops around AGI. If it’s driven underground it becomes much less likely to happen given the resources required.
So my thought on this is I think of flamethrowers and gas shells and the worst ww1 battlefields. I am not sure what taboo humans won’t violate in order to win.
This isn’t war though. What are some peace-time examples of taboo violations (especially state-sanctioned ones)? I can only really think of North Korea and a handful of other pariah states (none of which would be capable of developing AGI).
This can be avoided with a treaty that requires full access given to international inspectors. This already happens with the IAEA and was set up even in the far greater tensions of the cold war. If someone like Iran tries to kick out the inspectors, everyone assumes they’re trying to develop nuclear weapons and takes serious action (harsh sanctions, airstrikes, even the threat of war).
If governments think of this as an existential threat, they should agree to it for the same reasons they did with the IAEA. And while there’s big incentives to defect (unless they have very high p(doom)), there is also the knowledge that kicking out inspectors will lead to potential war and their rivals defecting too.
Sure. In practice there’s the national sovereignty angle though. This just devolves to each party “complies” with the agreement, violating it in various ways. Too much incentive to defect.
The US government just never audits its secret national labs, China just never checks anything, Israel just openly decides they can’t afford to comply at all etc. Everyone claims to be in compliance.
Really depends on how much of a taboo develops around AGI. If it’s driven underground it becomes much less likely to happen given the resources required.
So my thought on this is I think of flamethrowers and gas shells and the worst ww1 battlefields. I am not sure what taboo humans won’t violate in order to win.
This isn’t war though. What are some peace-time examples of taboo violations (especially state-sanctioned ones)? I can only really think of North Korea and a handful of other pariah states (none of which would be capable of developing AGI).
This can be avoided with a treaty that requires full access given to international inspectors. This already happens with the IAEA and was set up even in the far greater tensions of the cold war. If someone like Iran tries to kick out the inspectors, everyone assumes they’re trying to develop nuclear weapons and takes serious action (harsh sanctions, airstrikes, even the threat of war).
If governments think of this as an existential threat, they should agree to it for the same reasons they did with the IAEA. And while there’s big incentives to defect (unless they have very high p(doom)), there is also the knowledge that kicking out inspectors will lead to potential war and their rivals defecting too.