This is a really interesting take, and I agree with many elements. There is one element I want to explore more, and one I’d like to contest.
Firstly, I find a lot of the acceleration vs deceleration debate to be mostly theoretical and academic—not unlike debating whether or not it is better to have tides or to stop them and have a still ocean. At the end of the day (four times a day in most places, if we’re being pedantic) the tide is still going to do its thing. It’s the same with technical progress. Could you make it harder to innovate and improve technology? Yes. But realistically speaking having a pause or freeze of status quo in anything approaching an effective manner is just not possible. It’s the same issue I had with signing an open letter declaring a freeze. You can get everyone in the nation to sign an open letter saying “Don’t commit crimes”, but that isn’t going to solve the crime problem. But that’s a bit of a tangent and I don’t want to hijack your post nor your comments with unrelated debate.
Secondly, I think the nuclear and AI debates are quite poor comparisons. Much of this is anecdotal, having worked in both industries in a regulation role. Firstly, the very high levels of anti-nuclear campaigning and risk aversion have resulted in nuclear energy being a very heavily (and effectively) regulated industry. If it was not for the amount of anti-nuclear sentiment, I don’t think we’d have that level of security today. I think that’s partly what makes it so safe. I agree when you discuss the risk tradeoffs between coal and nuclear that it’s not as clear-cut as may be imagined, but I don’t think it supports the core argument very well. Also, nuclear energy and AI are such different industries to undertake risk reduction in—mostly because of the leverages of control you have through licensing, resources, and capital. However, this may be because of the aforementioned lobbying resulting in very burdensome regulation and perhaps AI will be similarly easy to regulate in future.
It’s also very possible that I’m misinterpreting your point, so please do let me know if that’s the case.
Ultimately I agree with your core point that this is a fallacy seen in much AI Safety reasoning, and that even stopping now would be shutting the stable door after the horse has bolted, but I think that there is a middle ground where speed of improvement and slower safeguards is a good way to lessen risk. I actually think nuclear energy is a good example of this, rather than a poor one.
First, to your second point, I agree that they aren’t comparable, so I don’t want to respond to your discussion. I was not, in this specific post, arguing that anything about safety in the two domains is comparable. The claim, which you agree to in the final paragraph, is that there is an underlying fallacy which is present both places.
However, returning to your first tangential point, the claim that the acceleration versus deceleration debate is theoretical and academic seems hard to support. Domains where everyone is dedicated to minimizing regulation and going full speed ahead are vastly different than those where people agree that significant care is needed, and where there is significant regulation and public debate. You seem to explicitly admit exactly this when you say that nuclear power is very different than AI because of the “very high levels of anti-nuclear campaigning and risk aversion”—that is, public pressure against nuclear seemed to have stopped the metaphorical tide. So I’m confused about your beliefs here.
No worries, there was always a chance I was misinterpreting the claim in that section. Happy for us to skip that.
For my second section I was talking more about stasis in the more full sense ie a pause in innovation in certain areas. Some are asking for full stasis for a period of time in the name of safety, others for a slow-down. I agree that safe stasis is a fallacy for the reasons I outlined, and agree with most of your points—particularly everything being a risk-risk tradeoff. I’m not entirely sold on the plausability of slowdowns or pauses from a logistical deployment perspective, which is where I think I got bogged down in the reeds in my response there.
This is a really interesting take, and I agree with many elements. There is one element I want to explore more, and one I’d like to contest.
Firstly, I find a lot of the acceleration vs deceleration debate to be mostly theoretical and academic—not unlike debating whether or not it is better to have tides or to stop them and have a still ocean. At the end of the day (four times a day in most places, if we’re being pedantic) the tide is still going to do its thing. It’s the same with technical progress. Could you make it harder to innovate and improve technology? Yes. But realistically speaking having a pause or freeze of status quo in anything approaching an effective manner is just not possible. It’s the same issue I had with signing an open letter declaring a freeze. You can get everyone in the nation to sign an open letter saying “Don’t commit crimes”, but that isn’t going to solve the crime problem. But that’s a bit of a tangent and I don’t want to hijack your post nor your comments with unrelated debate.
Secondly, I think the nuclear and AI debates are quite poor comparisons. Much of this is anecdotal, having worked in both industries in a regulation role. Firstly, the very high levels of anti-nuclear campaigning and risk aversion have resulted in nuclear energy being a very heavily (and effectively) regulated industry. If it was not for the amount of anti-nuclear sentiment, I don’t think we’d have that level of security today. I think that’s partly what makes it so safe. I agree when you discuss the risk tradeoffs between coal and nuclear that it’s not as clear-cut as may be imagined, but I don’t think it supports the core argument very well. Also, nuclear energy and AI are such different industries to undertake risk reduction in—mostly because of the leverages of control you have through licensing, resources, and capital. However, this may be because of the aforementioned lobbying resulting in very burdensome regulation and perhaps AI will be similarly easy to regulate in future.
It’s also very possible that I’m misinterpreting your point, so please do let me know if that’s the case.
Ultimately I agree with your core point that this is a fallacy seen in much AI Safety reasoning, and that even stopping now would be shutting the stable door after the horse has bolted, but I think that there is a middle ground where speed of improvement and slower safeguards is a good way to lessen risk. I actually think nuclear energy is a good example of this, rather than a poor one.
First, to your second point, I agree that they aren’t comparable, so I don’t want to respond to your discussion. I was not, in this specific post, arguing that anything about safety in the two domains is comparable. The claim, which you agree to in the final paragraph, is that there is an underlying fallacy which is present both places.
However, returning to your first tangential point, the claim that the acceleration versus deceleration debate is theoretical and academic seems hard to support. Domains where everyone is dedicated to minimizing regulation and going full speed ahead are vastly different than those where people agree that significant care is needed, and where there is significant regulation and public debate. You seem to explicitly admit exactly this when you say that nuclear power is very different than AI because of the “very high levels of anti-nuclear campaigning and risk aversion”—that is, public pressure against nuclear seemed to have stopped the metaphorical tide. So I’m confused about your beliefs here.
No worries, there was always a chance I was misinterpreting the claim in that section. Happy for us to skip that.
For my second section I was talking more about stasis in the more full sense ie a pause in innovation in certain areas. Some are asking for full stasis for a period of time in the name of safety, others for a slow-down. I agree that safe stasis is a fallacy for the reasons I outlined, and agree with most of your points—particularly everything being a risk-risk tradeoff. I’m not entirely sold on the plausability of slowdowns or pauses from a logistical deployment perspective, which is where I think I got bogged down in the reeds in my response there.