I agree that I’d love to see more work on this! (And I agree that the last story I talk about, of a very fast takeoff AI system with particularly advanced capabilities, seems unlikely to me—although others disagree, and think this “worst case” is also the most likely outcome.)
It’s worth noting again though that any particular story is unlikely to be correct. We’re trying to forecast the future, and good ways of forecasting should feel uncertain at the end, because we don’t know what the future will hold. Also, good work on this will (in my opinion) give us ideas about what many possible scenarios will look like . This sort of work (e.g. the first half of this article, rather than the second), often feels less concrete, but is, I think, more likely to be correct—and can inform actions that target many possible scenarios rather than one single unlikely event.
All that said, I’m excited to see work like OpenPhil’s nearcasting project which I find particularly clarifying and which will, I hope, improve our ability to prevent a catastrophe.
I agree that I’d love to see more work on this! (And I agree that the last story I talk about, of a very fast takeoff AI system with particularly advanced capabilities, seems unlikely to me—although others disagree, and think this “worst case” is also the most likely outcome.)
It’s worth noting again though that any particular story is unlikely to be correct. We’re trying to forecast the future, and good ways of forecasting should feel uncertain at the end, because we don’t know what the future will hold. Also, good work on this will (in my opinion) give us ideas about what many possible scenarios will look like . This sort of work (e.g. the first half of this article, rather than the second), often feels less concrete, but is, I think, more likely to be correct—and can inform actions that target many possible scenarios rather than one single unlikely event.
All that said, I’m excited to see work like OpenPhil’s nearcasting project which I find particularly clarifying and which will, I hope, improve our ability to prevent a catastrophe.