Suppose Xi Jinping declares tomorrow that he he believed AI was an imminent existential threat. He then makes an ultimatum to the US: dismantle OpenAI and ban AI research within 6 months, or China will launch airstrikes on silicon valley datacentres.
I wasn’t imagining Eliezer was proposing that the government immediately threaten nuclear strikes unless datacenters are taken down, I was imagining instead that he was proposing governments make their top priority “make sure datacenters are taken down” and then from that, follow whatever level of escalation was needed for that to happen (basically, once there was a clear policy, begin with asking nicely, then use diplomatic efforts, then economic efforts like sanctions, then make military threats, then, if needed, follow through on those military threats, using both carrots and sticks along this escalatory ladder). Of course, it would be foolish to begin with the highest rung on the ladder, and it would obviously be much preferable to never reach the higher levels.
2. If the world takes AI risk seriously, do we need threats?
Hopefully not! No one is saying “we should use military threats, whether they’re needed or not”, but instead that the government should be willing to escalate to that level if necessary. I don’t think it’s crazy to imagine a world where most countries take the risk seriously, but a few rogue nations continue on ahead despite the efforts of the rest of the world to stop them.
3. Don’t do morally wrong things
I don’t think it’s necessarily immoral for governments to enforce international treaties against WMDs (and similar) with the use of force once other avenues have been exhausted. I stand by this statement even if the enforcement is against a rogue nation that hasn’t signed the treaty themselves. Having said that, the specifics matter here – there’s obviously a history of governments using the false pretense of WMDs to carry out immoral military actions for other purposes (like geopolitical struggles).
4. Nuclear exchanges could be part of a rogue AI plan
If we are in a world which already has a scheming AI that wants to kill us all, then “start a nuclear war” seems like a fairly obvious move for it to make, assuming it has planned ahead.
I find it very unlikely that nuclear exchange would counterfactually be part of a rogue AI plan for human extinction. We’re imagining an AI (or group of AIs) that can out-scheme all of humanity combined and that can continue to support itself in the absence of humanity, yet the only way it can do its job involves a nuclear exchange, and one based specifically on the assumed existence of GPU clusters?
I wasn’t imagining Eliezer was proposing that the government immediately threaten nuclear strikes unless datacenters are taken down, I was imagining instead that he was proposing governments make their top priority “make sure datacenters are taken down” and then from that, follow whatever level of escalation was needed for that to happen (basically, once there was a clear policy, begin with asking nicely, then use diplomatic efforts, then economic efforts like sanctions, then make military threats, then, if needed, follow through on those military threats, using both carrots and sticks along this escalatory ladder). Of course, it would be foolish to begin with the highest rung on the ladder, and it would obviously be much preferable to never reach the higher levels.
Hopefully not! No one is saying “we should use military threats, whether they’re needed or not”, but instead that the government should be willing to escalate to that level if necessary. I don’t think it’s crazy to imagine a world where most countries take the risk seriously, but a few rogue nations continue on ahead despite the efforts of the rest of the world to stop them.
I don’t think it’s necessarily immoral for governments to enforce international treaties against WMDs (and similar) with the use of force once other avenues have been exhausted. I stand by this statement even if the enforcement is against a rogue nation that hasn’t signed the treaty themselves. Having said that, the specifics matter here – there’s obviously a history of governments using the false pretense of WMDs to carry out immoral military actions for other purposes (like geopolitical struggles).
I find it very unlikely that nuclear exchange would counterfactually be part of a rogue AI plan for human extinction. We’re imagining an AI (or group of AIs) that can out-scheme all of humanity combined and that can continue to support itself in the absence of humanity, yet the only way it can do its job involves a nuclear exchange, and one based specifically on the assumed existence of GPU clusters?