Hi Marcus, I’m in the mood for a bit of debate, so I’m going to take a stab at responding to all four of your points :)
LMK what you think!
1. This is an argument against a pause policy not the Pause org or a Pause movement. I think discerning funders need to see the differences. Especially if you have thinking on the margin.
2. “Pausing AI development for any meaningful amount of time is incredibly unlikely to occur.” < I think anything other than AGI in less than 10 years is unlikely to occur, but that isn’t a good argument to not work on Safety. Scale and neglectedness matter, as well as tractibility!
”they mainly seem to do a bunch of protesting where they do stuff like call Sam Altman and Dario Amodei evil.”
- Can you show evidence of this please?
3. “Pause AI, the organization, does, frankly, juvenile stunts that make EA/AI safety advocates look less serious.”
- Samesies—can you provide evidence please?
In fact, this whole point seems pretty unjustified. It seems you’re basically arguing that advocacy doesn’t work? Is that correct?
4. “Pause AI’s premise … only makes sense if you have extremely high AI extinction probabilities”
Can you justify this point please? I think it is interesting but it isn’t really explained.
I don’t think there is a need for me to show the relationship here.
2⁄3. https://youtu.be/T-2IM9P6tOs?si=uDiJXEqq8UJ63Hy2 this video came up as the first search result when i searched “pause ai protest” on youtube.
In it, the chant things like “open ai sucks! Anthropic sucks! Mistral sucks!” And “Demis Hassabis, reckless! Darío amodei reckless”
I agree that working on safety is a key moral priority. But working on safety looks a lot more like the things I linked to in #3. That’s what doing work looks like.
This seems to be what a typical protest looks like. I’ve seen videos of others. I consider these to be juvenile and unserious and unlikely to build necessary bridged to accomplish outcomes. I’ll let others form their opinions.
Correct, I potentially misremembered. the actual things they definitely say, at least in this video are “open ai sucks! Anthropic sucks! Mistral sucks!” And “Demis Hassabis, reckless! Darío amodei reckless”
I would submit that I am at the very least directionally correct.
“Demis Hassabis, reckless!” honestly feels to me like a pretty tame protest chant. I did a Google search for “protest” and this was the first result. Signs are things like “one year of genocide funded by UT” which seems both substantially more extreme and less epistemically valid than calling Demis “reckless.”
My sense from your other points is that you just don’t actually want pause AI to accomplish their goals, so it’s kind of over-determined for you, but if I wanted to tell a story about how a grassroots movement successfully got a international pause on AI, various people chanting that the current AI development process is reckless seems pretty fine to me?
Actually, I’m uncertain if pausing AI is a good idea and I wish the Pause AI people had a bit more uncertainty (on both their “p(doom)” and on whether pausing AI is a good policy) as well. I look at people who have 90%+ p(doom) as, at the very least, uncalibrated, the same way I look at the people who are dead certain that AI is going to go positively brilliant and that we should be racing ahead as fast as possible. It’s as if both of them aren’t doing any/enough reading of history. In the case of my tribe
I would submit that this kind of protesting, including/especially the example you posted makes your cause seem dumb/unnuanced/ridiculous to the onlookers who are indifferent/know little.
Last, I was just responding to the prompt “What are some criticisms of PauseAI?”. It’s not exactly the place for a “fair and balanced view” but also, I think it is far more important to critique your own side than the opposite side since you speak the same language as your own team so they will actually listen to you.
What is a reasonable p(doom|ASI) to have to not be concluding that pausing AI is a good idea? Or—what % chance of death are you personally willing to accept for a shot at immortality/utopia? Would it be the same if it was framed in terms of a game of Russian Roulette?
Hi Marcus, I’m in the mood for a bit of debate, so I’m going to take a stab at responding to all four of your points :)
LMK what you think!
1. This is an argument against a pause policy not the Pause org or a Pause movement. I think discerning funders need to see the differences. Especially if you have thinking on the margin.
2. “Pausing AI development for any meaningful amount of time is incredibly unlikely to occur.” < I think anything other than AGI in less than 10 years is unlikely to occur, but that isn’t a good argument to not work on Safety. Scale and neglectedness matter, as well as tractibility!
”they mainly seem to do a bunch of protesting where they do stuff like call Sam Altman and Dario Amodei evil.”
- Can you show evidence of this please?
3. “Pause AI, the organization, does, frankly, juvenile stunts that make EA/AI safety advocates look less serious.”
- Samesies—can you provide evidence please?
In fact, this whole point seems pretty unjustified. It seems you’re basically arguing that advocacy doesn’t work? Is that correct?
4. “Pause AI’s premise … only makes sense if you have extremely high AI extinction probabilities”
Can you justify this point please? I think it is interesting but it isn’t really explained.
I don’t think there is a need for me to show the relationship here.
2⁄3. https://youtu.be/T-2IM9P6tOs?si=uDiJXEqq8UJ63Hy2 this video came up as the first search result when i searched “pause ai protest” on youtube. In it, the chant things like “open ai sucks! Anthropic sucks! Mistral sucks!” And “Demis Hassabis, reckless! Darío amodei reckless”
I agree that working on safety is a key moral priority. But working on safety looks a lot more like the things I linked to in #3. That’s what doing work looks like.
This seems to be what a typical protest looks like. I’ve seen videos of others. I consider these to be juvenile and unserious and unlikely to build necessary bridged to accomplish outcomes. I’ll let others form their opinions.
The provided source doesn’t show PauseAI affiliated people calling Sam Altman and Dario Amodei evil.
Correct, I potentially misremembered. the actual things they definitely say, at least in this video are “open ai sucks! Anthropic sucks! Mistral sucks!” And “Demis Hassabis, reckless! Darío amodei reckless”
I would submit that I am at the very least directionally correct.
“Demis Hassabis, reckless!” honestly feels to me like a pretty tame protest chant. I did a Google search for “protest” and this was the first result. Signs are things like “one year of genocide funded by UT” which seems both substantially more extreme and less epistemically valid than calling Demis “reckless.”
My sense from your other points is that you just don’t actually want pause AI to accomplish their goals, so it’s kind of over-determined for you, but if I wanted to tell a story about how a grassroots movement successfully got a international pause on AI, various people chanting that the current AI development process is reckless seems pretty fine to me?
Actually, I’m uncertain if pausing AI is a good idea and I wish the Pause AI people had a bit more uncertainty (on both their “p(doom)” and on whether pausing AI is a good policy) as well. I look at people who have 90%+ p(doom) as, at the very least, uncalibrated, the same way I look at the people who are dead certain that AI is going to go positively brilliant and that we should be racing ahead as fast as possible. It’s as if both of them aren’t doing any/enough reading of history. In the case of my tribe
I would submit that this kind of protesting, including/especially the example you posted makes your cause seem dumb/unnuanced/ridiculous to the onlookers who are indifferent/know little.
Last, I was just responding to the prompt “What are some criticisms of PauseAI?”. It’s not exactly the place for a “fair and balanced view” but also, I think it is far more important to critique your own side than the opposite side since you speak the same language as your own team so they will actually listen to you.
What is a reasonable p(doom|ASI) to have to not be concluding that pausing AI is a good idea? Or—what % chance of death are you personally willing to accept for a shot at immortality/utopia? Would it be the same if it was framed in terms of a game of Russian Roulette?