Making the writing easy for myself: What’s your response to Carl Shulman’s take? Which is, that pushing for a pause too early might spoil the chance of getting people to agree to a pause when it would matter the most: at pivotal points where AI improvement is happening tremendously quickly. Carl Shulman on the 80k podcast.
Short answer I think trying to time this is too galaxy-brained. I think getting the meme of Pause out there ASAP is good because it pushes the Overton window and it gives people longer to chew on it. If and when warning shots occur, they will mainly advance Pause if people already had the idea that Pause would combat things like the warning shot happening before they happened.
I think takes that rely on saving up some kind of political capital and deploying it at the perfect time are generally wrong. PauseAI will gain more capital with more time and conversation, not use it up.
I’m not Holly, but my response is that getting a pause now is likely to increase, rather than decrease, the chance of getting future pauses. Quoting Evan Hubinger (2022):
In the theory of political capital, it is a fairly well-established fact that ‘Everybody Loves a Winner.’ That is: the more you succeed at leveraging your influence to get things done, the more influence you get in return. This phenomenon is most thoroughlystudied in the context of the ability of U.S. presidents to get their agendas through Congress—contrary to a naive model that might predict that legislative success uses up a president’s influence, what is actually found is the opposite: legislative success engenders future legislative success, greater presidential approval, and long-term gains for the president’s party.
I think many people who think about the mechanics of leveraging influence don’t really understand this phenomenon and conceptualize their influence as a finite resource to be saved up over time so it can all be spent down when it matters most. But I think that is just not how it works: if people see you successfully leveraging influence to change things, you become seen as a person who has influence, has the ability to change things, can get things done, etc. in a way that gives you more influence in the future, not less.
My sense is that this is a pretty major crux between my and Carl’s views.
Also not Holly, but another response might be the following:
Pausing in the very near future without a rise in political salience is just very very unlikely. The pause movement getting large influence is unlikely without a similar rise in political salience.
If a future rise in political salience occurs, this is likely an approximation of a ‘pivotal point’ (and if its not, well, policymakers are unlikely to agree to a pause at a pivotal point anyway)
Thus, what advocacy now is actually doing predominantly is creating the groundwork for a movement/idea that can be influential when the time comes.
I think this approach runs real risks, which I’d be happy to discuss, but also strikes me as an important response to the Shulman take.
Making the writing easy for myself: What’s your response to Carl Shulman’s take? Which is, that pushing for a pause too early might spoil the chance of getting people to agree to a pause when it would matter the most: at pivotal points where AI improvement is happening tremendously quickly. Carl Shulman on the 80k podcast.
You may have responded to this before. Feel free to provide a link.
This page is giving me a 404 right now: https://pauseai.info/mitigating-pause-failures
Short answer I think trying to time this is too galaxy-brained. I think getting the meme of Pause out there ASAP is good because it pushes the Overton window and it gives people longer to chew on it. If and when warning shots occur, they will mainly advance Pause if people already had the idea that Pause would combat things like the warning shot happening before they happened.
I think takes that rely on saving up some kind of political capital and deploying it at the perfect time are generally wrong. PauseAI will gain more capital with more time and conversation, not use it up.
I’m not Holly, but my response is that getting a pause now is likely to increase, rather than decrease, the chance of getting future pauses. Quoting Evan Hubinger (2022):
My sense is that this is a pretty major crux between my and Carl’s views.
Also not Holly, but another response might be the following:
Pausing in the very near future without a rise in political salience is just very very unlikely. The pause movement getting large influence is unlikely without a similar rise in political salience.
If a future rise in political salience occurs, this is likely an approximation of a ‘pivotal point’ (and if its not, well, policymakers are unlikely to agree to a pause at a pivotal point anyway)
Thus, what advocacy now is actually doing predominantly is creating the groundwork for a movement/idea that can be influential when the time comes.
I think this approach runs real risks, which I’d be happy to discuss, but also strikes me as an important response to the Shulman take.