Strong agreement that a global moratorium would be great.
I’m unsure if aiming for a global moratorium is the best thing to aim for rather than a slowing of the race-like behaviour—maybe a relevant similar case is whether to aim directly for the abolition of factory farms or just incremental improvements in welfare standards.
Loudly and publicly calling for a global moratorium should have the effect of slowing down race-like behaviour, even if it is ultimately unsuccessful. We can at least buy some more time, it’s not all or nothing in that sense. And more time can be used to buy yet more time, etc.
Factory farming is an interesting analogy, but the trade-off is different. You can think about whether abolitionism or welfarism has higher EV over the long term, but the stakes aren’t literally the end of the world if factory farming continues to gain power for 5-15 more years (i.e. humanity won’t end up in them).
The linked post is great, thanks for the reminder of it (and good to see it so high up the All Time top LW posts now). Who wants to start the institution lc talks about at the end? Who wants to devote significant resources to working on convincing AGI capabilities researchers to stop?
Isn’t it possible that calling for a complete stop to AI development actually counterfactually speeds up AI development?
The scenario I’m thinking of is something like:
There’s a growing anti-AI movement calling for a complete stop
A lot of people in that movement are ignorant about AI, and about the nature AI risks
It’s therefore easy for pro-AI people to dismiss these concerns, because the reasons given for the stop are in fact wrong/bad
Any other, well-grounded calls for AI slowdown aren’t given the time of day, because they are assumed to be the same as the others
Rather than thoughtful debate, the discourse turns into just attacking the other group
I’m not sure how exactly you’re proposing to advocate for a complete stop, but my worry would be that coming across as alarmist and not being able to give compelling specific reasons that AI poses a serious existential threat would poison the well.
I think it’s great that you’re trying to act seriously on your beliefs, Greg, but I am worried about a dynamic like this.
Strong agreement that a global moratorium would be great.
I’m unsure if aiming for a global moratorium is the best thing to aim for rather than a slowing of the race-like behaviour—maybe a relevant similar case is whether to aim directly for the abolition of factory farms or just incremental improvements in welfare standards.
This post from last year—What an actually pessimistic containment strategy looks like - has some good discussion on the topic of slowing down AGI research.
Loudly and publicly calling for a global moratorium should have the effect of slowing down race-like behaviour, even if it is ultimately unsuccessful. We can at least buy some more time, it’s not all or nothing in that sense. And more time can be used to buy yet more time, etc.
Factory farming is an interesting analogy, but the trade-off is different. You can think about whether abolitionism or welfarism has higher EV over the long term, but the stakes aren’t literally the end of the world if factory farming continues to gain power for 5-15 more years (i.e. humanity won’t end up in them).
The linked post is great, thanks for the reminder of it (and good to see it so high up the All Time top LW posts now). Who wants to start the institution lc talks about at the end? Who wants to devote significant resources to working on convincing AGI capabilities researchers to stop?
Isn’t it possible that calling for a complete stop to AI development actually counterfactually speeds up AI development?
The scenario I’m thinking of is something like:
There’s a growing anti-AI movement calling for a complete stop
A lot of people in that movement are ignorant about AI, and about the nature AI risks
It’s therefore easy for pro-AI people to dismiss these concerns, because the reasons given for the stop are in fact wrong/bad
Any other, well-grounded calls for AI slowdown aren’t given the time of day, because they are assumed to be the same as the others
Rather than thoughtful debate, the discourse turns into just attacking the other group
I’m not sure how exactly you’re proposing to advocate for a complete stop, but my worry would be that coming across as alarmist and not being able to give compelling specific reasons that AI poses a serious existential threat would poison the well.
I think it’s great that you’re trying to act seriously on your beliefs, Greg, but I am worried about a dynamic like this.
Well I’ve articulated what I think are compelling, specific reasons, that AI poses a serious existential threat in my new post: AGI rising: why we are in a new era of acute risk and increasing public awareness, and what to do now. Hope this can positively impact the public discourse toward informed debate. (And action!)
Yes, thank you for that! I’m probably going to write an object level comment there.