I both think that if it’s possible to stop the building of dangerously large models via international regulation, that would be bad because of tyranny risk, and I also think that we very likely can’t use international regulation to stop building these things, so that any local pauses are not going to have their intended effects and will have a lot of unintended net-negative effects.
(Also, reread my piece—I call for action to regulate and stop larger and more dangerous models immediately as a prelude to a global moratorium. I didn’t say “wait a while, then impose a pause for a while in a few places.”)
This really sounds like you are committing the fallacy I was worried about earlier on. I just don’t agree that you will actually get the global moratorium. I am fully aware of what your position is.
I think that you’re claiming something much stronger than “we very likely can’t use international regulation to stop building these things”—you’re claiming that international regulation won’t even be useful to reduce risk by changing incentives. And you’ve already agreed that it’s implausible that these efforts would lead to tyranny, you think they will just fail.
But how they fail matters—there’s a huge difference between something like the NPT, which was mostly effective, and something like the Kellogg-Briand Pact of 1928, which was ineffective but led to a huge change, versus… I don’t know, I can’t really think of many examples of treaties or treaty negotiations that backfired, even though most fail to produce exactly what they hoped. (I think there’s a stronger case to make that treaties can prevent the world from getting stronger treaties later, but that’s not what you claim.)
And you’ve already agreed that it’s implausible that these efforts would lead to tyranny, you think they will just fail.
I think that conditional on the efforts working, the chance of tyranny is quite high (ballpark 30-40%). I don’t think they’ll work, but if they do, it seems quite bad.
And since I think x-risk from technical AI alignment failure is in the 1-2% range, the risk of tyranny is the dominant effect of “actually enforced global AI pause” in my EV calculation, followed by the extra fast takeoff risks, and then followed by “maybe we get net positive alignment research.”
Conditional on “the efforts” working is hooribly underspecified. A global governance mechanism run by a new extranational body with military powers monitoring and stopping production of GPUs, or a standard treaty with a multi-party inspection regime?
I’m not conditioning on the global governance mechanism— I assign nonzero probability mass to the “standard treaty” thing— but I think in fact you would very likely need global governance, so that is the main causal mechanism through which tyranny happens in my model
My opposition is disjunctive!
I both think that if it’s possible to stop the building of dangerously large models via international regulation, that would be bad because of tyranny risk, and I also think that we very likely can’t use international regulation to stop building these things, so that any local pauses are not going to have their intended effects and will have a lot of unintended net-negative effects.
This really sounds like you are committing the fallacy I was worried about earlier on. I just don’t agree that you will actually get the global moratorium. I am fully aware of what your position is.
I think that you’re claiming something much stronger than “we very likely can’t use international regulation to stop building these things”—you’re claiming that international regulation won’t even be useful to reduce risk by changing incentives. And you’ve already agreed that it’s implausible that these efforts would lead to tyranny, you think they will just fail.
But how they fail matters—there’s a huge difference between something like the NPT, which was mostly effective, and something like the Kellogg-Briand Pact of 1928, which was ineffective but led to a huge change, versus… I don’t know, I can’t really think of many examples of treaties or treaty negotiations that backfired, even though most fail to produce exactly what they hoped. (I think there’s a stronger case to make that treaties can prevent the world from getting stronger treaties later, but that’s not what you claim.)
I think that conditional on the efforts working, the chance of tyranny is quite high (ballpark 30-40%). I don’t think they’ll work, but if they do, it seems quite bad.
And since I think x-risk from technical AI alignment failure is in the 1-2% range, the risk of tyranny is the dominant effect of “actually enforced global AI pause” in my EV calculation, followed by the extra fast takeoff risks, and then followed by “maybe we get net positive alignment research.”
Conditional on “the efforts” working is hooribly underspecified. A global governance mechanism run by a new extranational body with military powers monitoring and stopping production of GPUs, or a standard treaty with a multi-party inspection regime?
I’m not conditioning on the global governance mechanism— I assign nonzero probability mass to the “standard treaty” thing— but I think in fact you would very likely need global governance, so that is the main causal mechanism through which tyranny happens in my model