Thanks. I think that your summary is great and touched on basically all the major points that I meant to address in my original Twitter thread. I just have one clarification.
[Caveat lector: It seems likely to me that I’m not grokking Matthew here, this section is rougher and more likely to mislead you]: Matthew claims that the there’s only so much budget of delays that humanity has. He argues that humanity should spend that budget later, when AI is closer to being an existential threat, rather than now.
I want to note that this argument was not spelled out very well, so I can understand why it might be confusing. I didn’t mean to make a strong claim about whether we have a fixed budget of delays; only that it’s possible. In fact, there seems to be a reason why we wouldn’t have a fixed budget, since delaying now might help us delay later.
Nonetheless, delaying now means that we get more overhang, which means that we might not be able to “stop” and move through later stages as slowly as we’re moving through our current stage of AI development. Even if the moratorium is lifted slowly, I think we’d still get less incremental progress later than if we had never had the moratorium to begin with, although justifying this claim would take a while, and likely requires a mathematical model of the situation. But this is essentially just a repetition of the point about overhangs stated in a different way, rather than a separate strong claim that we only get a fixed budget of delays.
Thanks. I think that your summary is great and touched on basically all the major points that I meant to address in my original Twitter thread. I just have one clarification.
I want to note that this argument was not spelled out very well, so I can understand why it might be confusing. I didn’t mean to make a strong claim about whether we have a fixed budget of delays; only that it’s possible. In fact, there seems to be a reason why we wouldn’t have a fixed budget, since delaying now might help us delay later.
Nonetheless, delaying now means that we get more overhang, which means that we might not be able to “stop” and move through later stages as slowly as we’re moving through our current stage of AI development. Even if the moratorium is lifted slowly, I think we’d still get less incremental progress later than if we had never had the moratorium to begin with, although justifying this claim would take a while, and likely requires a mathematical model of the situation. But this is essentially just a repetition of the point about overhangs stated in a different way, rather than a separate strong claim that we only get a fixed budget of delays.