Warning shots/accidents are normally discussed in the frame of generating political will, by convincing a previously unpersuaded public or policymakers that AI is unsafe and action must be taken.
I think this is a mistake.
Accidents (which might be relatively small-scale), in AI as in other fields, are useful mainly for generating real-world, non-hypothetical failure cases in all their intricate detail, thereby yielding a model organism which can be studied by engineers (and hopefully reproduced in a controlled manner) to better understand both the circumstances in which such scenarios might arise, and countermeasures to prevent them.
This is analogous to how aircraft accidents are investigated in depth by the NTSB so as to learn how to prevent similar accidents. There’s already political will to make aircraft safe, but there’s only so much that can be done from the ivory tower without real-world experience.
The choices are:
Stop AI development permanently.
Pause AI temporarily until we make it safe.
Muddle through.
The printing press and electricity were existentially dangerous technologies, because they enabled everything that came after, including AI. When those technologies were developed, however, the world wasn’t globalized enough, nor were nations powerful enough, that a permanent stop button could have been pressed. By contrast, perhaps a permanent “stop AI” button could be pressed today, however I don’t see any way of doing so short of entrenching a permanent totalitarian state.
So that leaves pausing until we make it safe, or muddling through.
But I think the aircraft accident analogy works quite well for AI: there’s only so much that safety research can do from the ivory tower without experience of AIs being used in the real world. So I think the “pause until we make it safe” option is illusory.
That leaves muddling through, as we’ve done with every technology before: We discover problems, hopefully at a small scale, and fix or mitigate them as they arise.
There are no guarantees, but I think it’s our best bet.
I notice someone added the “AI risk skepticism” tag to this post. That seems reasonable, because I do express skepticism of the extreme certainty that some have about misalignment risk, even though I think that risk is real.
But my main goal with this post is to argue for, not against, the reality of a certain type of catastrophic AI risk that I believe many are largely ignoring and inadvertently worsening: the risk that AI is a powerful technology which can enable a permanent authoritarian lock-in over the human future, and that draconian controls over the enabling resources of that technology, especially in a time of rapid democratic backsliding, could help bring into existence.