I appreciate Eliezer’s honesty and consistency in what he is is calling for. This approach makes sense if you believe, as Eliezer does, that p(doom | business as usual)>99%. Then it is worth massively increasing the risk of a nuclear war. If you believe, as I do and as most AI experts do, that p(doom | business as usual) <20%, this plan is absolutely insane.
This line of thinking is becoming more and more common in EA. It is going to get us all killed if it has any traction. No, the U.S. should not be willing to bomb Chinese data centers and risk a global nuclear war. No, repeatedly bombing China for pursing something that is a central goal of the CCP that has dangers that are completely illegible to 90% of the population is not a small, incremental risk of nuclear war on the scale of aiding Ukraine as some other commenters are suggesting. This is insane.
By all means, I support efforts for international treaties. Bombing Chinese data centers is suicidal and we all know it.
I say this all as someone who is genuinely frightened of AGI. It might well kill us, but not as quickly or surely as implementing this strategy will.
Edited to reflect that upon further thought, I probably do not support bombing the data centers of less powerful countries either.
Why do you assume that China/the CCP (or any other less powerful countries) won’t wake up to the risk of AGI? They aren’t suicidal. The way I interpret “rogue data centre” is a cavalier non-state actor (or even a would-be world-ending terrorist cell).
Unless the non-state actor is operating off a ship in international waters, it’s operating within a nation-state’s boundaries, and bombing it would be a serious incursion on that nation-state’s territorial sovereignty. There’s a reason such incursions against a nuclear state have been off the table except in the most dire of circumstances.
The possibility of some actor having the financial and intellectual resources necessary to develop AGI without the acquiescence of the nation within which operates seems rather remote. And the reference elsewhere to nuclear options probably colors this statement—why discuss that if the threat model is some random terrorist cell?
Ok, I guess it depends on what level the “acquiescence” is. I’d hope that diplomacy would be successful in nearly all cases here, with the nuclear state reining in the rogue data centre inside its borders without any outside coalition resorting to an airstrike.
You mention “except in the most dire of circumstances”—these would be the absolute most dire circumstances imaginable. More dire, in fact, than anything in recorded history—the literal end of the world at stake.
I hope they do wake up to the danger, and I am all for trying to negotiate treaties! It’s possible I am misinterpreting what EY means by “rogue data centers.” To clarify, the specific thing I am calling insane is the idea that the U.S. or NATO should under (almost) any circumstance bomb data centers inside other nuclear powers.
I don’t think this is insane, and I think <20% is probably too low a threshold to carry the case—a 15% risk of extinction from AGI would mean we should be drastically, drastically more scared of AGI than of nuclear war.
What percentage chance would you estimate of a large scale nuclear war conditional on the U.S. bombing a Chinese data center? What percentage of the risk from agi do you think this strategy reduces?
I appreciate Eliezer’s honesty and consistency in what he is is calling for. This approach makes sense if you believe, as Eliezer does, that p(doom | business as usual)>99%. Then it is worth massively increasing the risk of a nuclear war. If you believe, as I do and as most AI experts do, that p(doom | business as usual) <20%, this plan is absolutely insane.
This line of thinking is becoming more and more common in EA. It is going to get us all killed if it has any traction. No, the U.S. should not be willing to bomb Chinese data centers and risk a global nuclear war. No, repeatedly bombing China for pursing something that is a central goal of the CCP that has dangers that are completely illegible to 90% of the population is not a small, incremental risk of nuclear war on the scale of aiding Ukraine as some other commenters are suggesting. This is insane.
By all means, I support efforts for international treaties. Bombing Chinese data centers is suicidal and we all know it.
I say this all as someone who is genuinely frightened of AGI. It might well kill us, but not as quickly or surely as implementing this strategy will.
Edited to reflect that upon further thought, I probably do not support bombing the data centers of less powerful countries either.
Why do you assume that China/the CCP (or any other less powerful countries) won’t wake up to the risk of AGI? They aren’t suicidal. The way I interpret “rogue data centre” is a cavalier non-state actor (or even a would-be world-ending terrorist cell).
Unless the non-state actor is operating off a ship in international waters, it’s operating within a nation-state’s boundaries, and bombing it would be a serious incursion on that nation-state’s territorial sovereignty. There’s a reason such incursions against a nuclear state have been off the table except in the most dire of circumstances.
The possibility of some actor having the financial and intellectual resources necessary to develop AGI without the acquiescence of the nation within which operates seems rather remote. And the reference elsewhere to nuclear options probably colors this statement—why discuss that if the threat model is some random terrorist cell?
Ok, I guess it depends on what level the “acquiescence” is. I’d hope that diplomacy would be successful in nearly all cases here, with the nuclear state reining in the rogue data centre inside its borders without any outside coalition resorting to an airstrike.
You mention “except in the most dire of circumstances”—these would be the absolute most dire circumstances imaginable. More dire, in fact, than anything in recorded history—the literal end of the world at stake.
I hope they do wake up to the danger, and I am all for trying to negotiate treaties!
It’s possible I am misinterpreting what EY means by “rogue data centers.” To clarify, the specific thing I am calling insane is the idea that the U.S. or NATO should under (almost) any circumstance bomb data centers inside other nuclear powers.
I don’t think this is insane, and I think <20% is probably too low a threshold to carry the case—a 15% risk of extinction from AGI would mean we should be drastically, drastically more scared of AGI than of nuclear war.
What percentage chance would you estimate of a large scale nuclear war conditional on the U.S. bombing a Chinese data center? What percentage of the risk from agi do you think this strategy reduces?