It’s very likely that whatever change that comes from AI development will be irreversible.
I think all actions are in a sense irreversible, but large changes tend to be less reversible than small changes. In this sense, the argument you gave seems reducible to “we should generally delay large changes to the world, to preserve option value”. Is that a reasonable summary?
In this case I think it’s just not obvious that delaying large changes is good. Would it have been good to delay the industrial revolution to preserve option value? I think this heuristic, if used in the past, would have generally demanded that we “pause” all sorts of social, material, and moral progress, which seems wrong.
I don’t think we would have been able to use the additional information we would have gained from delaying the industrial revolution but I think if we could have the answer might be “yes”. It’s easy to see in hindsight that it went well overall, but that doesn’t mean that the correct ex ante attitude shouldn’t have been caution!
I think all actions are in a sense irreversible, but large changes tend to be less reversible than small changes. In this sense, the argument you gave seems reducible to “we should generally delay large changes to the world, to preserve option value”. Is that a reasonable summary?
In this case I think it’s just not obvious that delaying large changes is good. Would it have been good to delay the industrial revolution to preserve option value? I think this heuristic, if used in the past, would have generally demanded that we “pause” all sorts of social, material, and moral progress, which seems wrong.
I don’t think we would have been able to use the additional information we would have gained from delaying the industrial revolution but I think if we could have the answer might be “yes”. It’s easy to see in hindsight that it went well overall, but that doesn’t mean that the correct ex ante attitude shouldn’t have been caution!