If you desperately wish we had more time to work on alignment, but also think a pause won’t make that happen or would have larger countervailing costs, then that would lead to an attitude like: “If only we had more time! But alas, a pause would only make things worse. Let’s talk about other ideas…” For my part, I definitely say things like that (see here).
However, Nora has sections claiming “alignment is doing pretty well” and “alignment optimism”, so I think it’s self-consistent for her to not express that kind of mood.
Insofar as Nora discusses nuances of overhang, it would be odd and annoying for that to not actually be cruxy for her (given that she doesn’t say something like this isn’t cruxy for me).
I was reading it as a kinda disjunctive argument. If Nora says that a pause is bad because of A and B, either of which is sufficient on its own from her perspective, then you could say “A isn’t cruxy for her” (because B is sufficient) or you could say “B isn’t cruxy for her” (because A is sufficient). Really, neither of those claims is accurate.
Oh well, whatever, I agree with you that the OP could have been clearer.
Yep it’s all meant to be disjunctive and yep it could have been clearer. FWIW this essay went through multiple major revisions and at one point I was trying to make the disjunctivity of it super clear but then that got de-prioritized relative to other stuff. In the future if/when I write about this I think I’ll be able to organize things significantly better
Hmm, AI safety is probably easy implies that slowing AI is lower-stakes but doesn’t obviously imply much about whether it’s net-positive. It’s not obvious to me what alignment optimism has to do with the pause debate, and I don’t think you discuss this.
It’s not obvious to me what alignment optimism has to do with the pause debate
Sorry, I thought it would be fairly obvious how it’s related. If you’re optimistic about alignment then the expected benefits you might hope to get out of a pause (whether or not you actually do get those benefits) are commensurately smaller, so the unintended consequences should have more relative weight in your EV calculation.
To be clear, I think slowing down AI in general, as opposed to the moratorium proposal in particular, is a more reasonable position that’s a bit harder to argue against. I do still think the overhang concerns apply in non-pause slowdowns but in a less acute manner.
Given alignment optimism, the benefits of pause are smaller—but the unintended consequences for alignment are smaller too. I guess alignment optimism suggests pause-is-bad if e.g. your alignment optimism is super conditional on smooth progress...
If you desperately wish we had more time to work on alignment, but also think a pause won’t make that happen or would have larger countervailing costs, then that would lead to an attitude like: “If only we had more time! But alas, a pause would only make things worse. Let’s talk about other ideas…” For my part, I definitely say things like that (see here).
However, Nora has sections claiming “alignment is doing pretty well” and “alignment optimism”, so I think it’s self-consistent for her to not express that kind of mood.
Insofar as Nora discusses nuances of overhang, it would be odd and annoying for that to not actually be cruxy for her (given that she doesn’t say something like this isn’t cruxy for me).
I was reading it as a kinda disjunctive argument. If Nora says that a pause is bad because of A and B, either of which is sufficient on its own from her perspective, then you could say “A isn’t cruxy for her” (because B is sufficient) or you could say “B isn’t cruxy for her” (because A is sufficient). Really, neither of those claims is accurate.
Oh well, whatever, I agree with you that the OP could have been clearer.
Yep it’s all meant to be disjunctive and yep it could have been clearer. FWIW this essay went through multiple major revisions and at one point I was trying to make the disjunctivity of it super clear but then that got de-prioritized relative to other stuff. In the future if/when I write about this I think I’ll be able to organize things significantly better
Hmm, AI safety is probably easy implies that slowing AI is lower-stakes but doesn’t obviously imply much about whether it’s net-positive. It’s not obvious to me what alignment optimism has to do with the pause debate, and I don’t think you discuss this.
Sorry, I thought it would be fairly obvious how it’s related. If you’re optimistic about alignment then the expected benefits you might hope to get out of a pause (whether or not you actually do get those benefits) are commensurately smaller, so the unintended consequences should have more relative weight in your EV calculation.
To be clear, I think slowing down AI in general, as opposed to the moratorium proposal in particular, is a more reasonable position that’s a bit harder to argue against. I do still think the overhang concerns apply in non-pause slowdowns but in a less acute manner.
Given alignment optimism, the benefits of pause are smaller—but the unintended consequences for alignment are smaller too. I guess alignment optimism suggests pause-is-bad if e.g. your alignment optimism is super conditional on smooth progress...
Could you say more about what you see as the practical distinction between a “slow down AI in general” proposal vs. a “pause” proposal?