[AI pause can be the best course of action even if the baseline risk of AI ruin were “only” 5%]
Some critics of pausing frontier AI research argue that AI pause would be an extreme measure (curtailing tech progress, which has historically led to so much good) that is only justifiable if we have very high confidence that the current path leads to ruin.
On the one hand, I feel like the baseline risk is indeed very high (I put it at 70% all-things-considered).
At the same time, I’m frustrated that there’s so much discussion of “is the baseline risk really that high?” compared to “is not pausing really the optimal way for us to go into this major civilizational transition?”
I feel like arguing about ruin levels can be distraction from what should be a similarly important crux. Something has gone wrong if people think pausing can only make sense if the risks of AI ruin are >50%.
The key question is: Will pausing reduce the risks?
Even if the baseline risk were “only” 5%, assuming we have a robust argument that pausing for (say) five years will reduce it to (say) 4%, that would clearly be good! (It would be very unfortunate for the people who will die preventable deaths in the next five years, but pausing would still be better on net, on these assumptions.)
So, what assumptions would have to be true that continuing ahead is better than pausing?
(Also, if someone is worried that there are negative side effects from pausing, such as that it’ll be politically/societally hard to re-start things later on after alignment research made breakthroughs or if some Western-values unaligned country is getting closer to building TAI themselves, that’s a discussion worth having! However, then we have to look at “the best implementation of pausing we can realistically get if we advocate for the smartest thing,” and not “a version of pausing that makes no attempts whatsoever to reduce bad side effects.”)
I think the factor missing here is the matter of when pushing for a pause is appropriate.
Like, imagine a (imo likely) scenario where a massive campaign gets off, with a lot of publicity behind it, to try and prevent GPT-5 from being released on existential risk grounds. It fails, and GPT-5 is released anyway , and literally nothing majorly bad happens. And then the same thing happens for gpt-6 and gpt-7.
In this scenario, the idea of pausing AI could easily become a laughing stock. Then when an actually dangerous AI comes out, the idea of pausing is still discredited, and you’re missing a tool when you really actually need it.
Even if I believed the risk of overall doom was 5% (way too high imo), I wouldn’t support the pause movement now, I’d want to wait on advocating a pause until there was a significant chance of imminent danger.
The point I wanted to make in the short form was directed at a particular brand of skeptic.
When I said,
Something has gone wrong if people think pausing can only make sense if the risks of AI ruin are >50%.
I didn’t mean to imply that anyone who opposes pausing would consider >50% ruin levels their crux.
Likewise, I didn’t mean to imply that “let’s grant 5% risk levels” is something that every skeptic would go along with (but good that your comment is making this explicit!).
For what it’s worth, if I had to give a range for how much I think people I, at the moment, epistemically respect to the highest extent possible, can disagree on this question today (June 2024), I would probably not include credences <<5% in that range (I’d maybe put it a more like 15-90%?). (This is of course subject to change if I encounter surprisingly good arguments for something outside the range.) But that’s a separate(!) discussion, separate from the conditional statement that I wanted to argue for in my short form. (Obviously, other people will draw the line elsewhere.)
On the 80k article, I think it aged less well than what one maybe could’ve written at the time, but it was written at a time when AI risk concerns still seemed fringe. So, just because it in my view didn’t age amazingly doesn’t mean that it was unreasonable at the time. At the time, I’d have probably called it “lower than what I would give, but seems within the range of what I consider reasonable.”
[AI pause can be the best course of action even if the baseline risk of AI ruin were “only” 5%]
Some critics of pausing frontier AI research argue that AI pause would be an extreme measure (curtailing tech progress, which has historically led to so much good) that is only justifiable if we have very high confidence that the current path leads to ruin.
On the one hand, I feel like the baseline risk is indeed very high (I put it at 70% all-things-considered).
At the same time, I’m frustrated that there’s so much discussion of “is the baseline risk really that high?” compared to “is not pausing really the optimal way for us to go into this major civilizational transition?”
I feel like arguing about ruin levels can be distraction from what should be a similarly important crux. Something has gone wrong if people think pausing can only make sense if the risks of AI ruin are >50%.
The key question is: Will pausing reduce the risks?
Even if the baseline risk were “only” 5%, assuming we have a robust argument that pausing for (say) five years will reduce it to (say) 4%, that would clearly be good! (It would be very unfortunate for the people who will die preventable deaths in the next five years, but pausing would still be better on net, on these assumptions.)
So, what assumptions would have to be true that continuing ahead is better than pausing?
(Also, if someone is worried that there are negative side effects from pausing, such as that it’ll be politically/societally hard to re-start things later on after alignment research made breakthroughs or if some Western-values unaligned country is getting closer to building TAI themselves, that’s a discussion worth having! However, then we have to look at “the best implementation of pausing we can realistically get if we advocate for the smartest thing,” and not “a version of pausing that makes no attempts whatsoever to reduce bad side effects.”)
I think the factor missing here is the matter of when pushing for a pause is appropriate.
Like, imagine a (imo likely) scenario where a massive campaign gets off, with a lot of publicity behind it, to try and prevent GPT-5 from being released on existential risk grounds. It fails, and GPT-5 is released anyway , and literally nothing majorly bad happens. And then the same thing happens for gpt-6 and gpt-7.
In this scenario, the idea of pausing AI could easily become a laughing stock. Then when an actually dangerous AI comes out, the idea of pausing is still discredited, and you’re missing a tool when you really actually need it.
Even if I believed the risk of overall doom was 5% (way too high imo), I wouldn’t support the pause movement now, I’d want to wait on advocating a pause until there was a significant chance of imminent danger.
Yeah, I agree. I wrote about timing considerations here; I agree this is an important part of the discussion.
“5%” is underestimating skepticism. Even those that publicized artificial intelligence risk didn’t claim much higher chances:
https://80000hours.org/articles/existential-risks/
The point I wanted to make in the short form was directed at a particular brand of skeptic.
When I said,
I didn’t mean to imply that anyone who opposes pausing would consider >50% ruin levels their crux.
Likewise, I didn’t mean to imply that “let’s grant 5% risk levels” is something that every skeptic would go along with (but good that your comment is making this explicit!).
For what it’s worth, if I had to give a range for how much I think people I, at the moment, epistemically respect to the highest extent possible, can disagree on this question today (June 2024), I would probably not include credences <<5% in that range (I’d maybe put it a more like 15-90%?). (This is of course subject to change if I encounter surprisingly good arguments for something outside the range.) But that’s a separate(!) discussion, separate from the conditional statement that I wanted to argue for in my short form. (Obviously, other people will draw the line elsewhere.)
On the 80k article, I think it aged less well than what one maybe could’ve written at the time, but it was written at a time when AI risk concerns still seemed fringe. So, just because it in my view didn’t age amazingly doesn’t mean that it was unreasonable at the time. At the time, I’d have probably called it “lower than what I would give, but seems within the range of what I consider reasonable.”