I’m surprised nobody has commented yet, and want to say that I really enjoyed and largely agree with this piece. The logic of needing to accelerate an AGI arms race to stay ahead of China is deeply flawed in ways that mirror unfortunate pathologies in the US foreign policy community, and IMO worsens US national security, for many of the reasons you mention.
Two questions for you:
How politically feasible is it to advance messaging along these lines, given the incoming administration’s tech optimism and zero-sum foreign policy mindset (and indeed, the rare bipartisan consensus on hawkishness towards China?) I could see a lot of folks in the EA community saying “You’re right, of course, but the train has left the station. We as a community lack the power to redirect policymakers’ incentives and perceived interests on this issue any time soon, and the timelines are getting shorter, so we don’t have time to try. Instead of marginalizing ourselves by trying to prevent an arms race that is by now inevitable and well underway, or push for collaborative international frameworks that MAGA has no interest in, it’d be more impactful to work within the existing incentives to slow down China and lobby for whatever marginal safety improvements we can.”
Why did you label the views we disagree with “AGI realism”? Is that the preferred title of their advocates or did you pick the word realism? I ask because I think much of the argument dramatizing the stakes of China getting this before us is linked with liberal internationalist mindsets that see the 21st century as a civilizational struggle between democracy and autocracy, and see AI as just one complicating wrinkle in that big-picture fight. Inversely, many of the voices calling for more restraint in US foreign policy (ex: abandoning hegemony and embracing multipolarity) call themselves realists, and see the path to peace as ensuring a stable and durable balance of power. So I think of it more as a debate between AI hawks and AI doves/restrainers, both of which could be either realists or something else.
1. It’s a great question. My biggest uncertainty here is whether the fact that tech is aligning on this rhetoric is purely reactive to Trump’s foreign policy goals, or whether this rhetoric from tech is actively shaping the frame here (e.g. through mouthpieces like Thiel/Musk). I don’t really have a sense of to what extent either is true, but my gut take is that it’s more the former. In which case while I think the EA community could work towards challenging this narrative within tech, I wouldn’t expect this to overpower the strong incentives to align with the administration as politics increasingly shifts from programmatic to personalist. Although at the very least, I think there should be more work done on the margins to mitigate what I think could be the harms of AGI realist rhetoric. Hedging strategies like maintaining attempts at collaboration (even if unlikely); more work done to reduce the odds that conflict goes nuclear; more work on even mitigating the harms of war, certainly a lot more work to mitigate autonomous AI risk, etc. But yeah, seems totally plausible the ship has sailed here.
2. AGI Realism is the title that Leopold Aschenbrenner gives to his views in the parting thoughts of his blog series, though I entirely agree with you here. I think if Aschenbrenner et al. truly held realist assumptions then they’d be sceptical of this idea that unipolarity through AI dominance could lead to the stable imposition of a safe AI order over China. If anything, I think realists who genuinely buy that superintelligence could be unlike anything ever seen before in terms of how much it could empower a hegemon would be concerned about levels of counterbalancing that are also unprecedented (i.e. not just China alone). I think if I’m a realist, I’m really looking for interstate checks and balances to constrain conflict. I think the exact danger with “AGI Realist” rhetoric is precisely that it’s actually built on the kind of flawed liberal institutionalism that underpins the failures of liberal interventionism.
I’m surprised nobody has commented yet, and want to say that I really enjoyed and largely agree with this piece. The logic of needing to accelerate an AGI arms race to stay ahead of China is deeply flawed in ways that mirror unfortunate pathologies in the US foreign policy community, and IMO worsens US national security, for many of the reasons you mention.
Two questions for you:
How politically feasible is it to advance messaging along these lines, given the incoming administration’s tech optimism and zero-sum foreign policy mindset (and indeed, the rare bipartisan consensus on hawkishness towards China?) I could see a lot of folks in the EA community saying “You’re right, of course, but the train has left the station. We as a community lack the power to redirect policymakers’ incentives and perceived interests on this issue any time soon, and the timelines are getting shorter, so we don’t have time to try. Instead of marginalizing ourselves by trying to prevent an arms race that is by now inevitable and well underway, or push for collaborative international frameworks that MAGA has no interest in, it’d be more impactful to work within the existing incentives to slow down China and lobby for whatever marginal safety improvements we can.”
Why did you label the views we disagree with “AGI realism”? Is that the preferred title of their advocates or did you pick the word realism? I ask because I think much of the argument dramatizing the stakes of China getting this before us is linked with liberal internationalist mindsets that see the 21st century as a civilizational struggle between democracy and autocracy, and see AI as just one complicating wrinkle in that big-picture fight. Inversely, many of the voices calling for more restraint in US foreign policy (ex: abandoning hegemony and embracing multipolarity) call themselves realists, and see the path to peace as ensuring a stable and durable balance of power. So I think of it more as a debate between AI hawks and AI doves/restrainers, both of which could be either realists or something else.
Thank you!
1. It’s a great question. My biggest uncertainty here is whether the fact that tech is aligning on this rhetoric is purely reactive to Trump’s foreign policy goals, or whether this rhetoric from tech is actively shaping the frame here (e.g. through mouthpieces like Thiel/Musk). I don’t really have a sense of to what extent either is true, but my gut take is that it’s more the former. In which case while I think the EA community could work towards challenging this narrative within tech, I wouldn’t expect this to overpower the strong incentives to align with the administration as politics increasingly shifts from programmatic to personalist. Although at the very least, I think there should be more work done on the margins to mitigate what I think could be the harms of AGI realist rhetoric. Hedging strategies like maintaining attempts at collaboration (even if unlikely); more work done to reduce the odds that conflict goes nuclear; more work on even mitigating the harms of war, certainly a lot more work to mitigate autonomous AI risk, etc. But yeah, seems totally plausible the ship has sailed here.
2. AGI Realism is the title that Leopold Aschenbrenner gives to his views in the parting thoughts of his blog series, though I entirely agree with you here. I think if Aschenbrenner et al. truly held realist assumptions then they’d be sceptical of this idea that unipolarity through AI dominance could lead to the stable imposition of a safe AI order over China. If anything, I think realists who genuinely buy that superintelligence could be unlike anything ever seen before in terms of how much it could empower a hegemon would be concerned about levels of counterbalancing that are also unprecedented (i.e. not just China alone). I think if I’m a realist, I’m really looking for interstate checks and balances to constrain conflict. I think the exact danger with “AGI Realist” rhetoric is precisely that it’s actually built on the kind of flawed liberal institutionalism that underpins the failures of liberal interventionism.