A warning shot is a global catastrophe that indirectly reduces existential risk by increasing concern about future catastrophes.
Terminology
The expression warning sign is sometimes used to describe any event that increases concern about a particular category of existential risk, regardless of whether the event itself constitutes a global catastrophe. For example, plausible candidates for an AI warning sign include a catastrophic failure by an AI system and public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.[1]
A related notion is that of a fire alarm, a warning sign that creates common knowledge that some technology—typically advanced artificial intelligence—actually poses an existential risk.[2]
Note, however, that both “warning shot” and “fire alarm” are sometimes used as synonyms for “warning sign”.[3][4]
Further reading
Beckstead, Nick (2015) The long-term significance of reducing global catastrophic risks, Open Philanthropy, August 13.
Carlsmith, Joseph (2021) Is power-seeking AI an existential risk?, Open Philanthropy, April, section 6.2.
Related entries
artificial intelligence | existential risk | global catastrophic risk
- ^
Hobson, Donald (2020) Comment on ‘What are the most plausible “AI Safety warning shot” scenarios?’, AI Alignment Forum, March 26.
- ^
Grace, Katja (2021) Beyond fire alarms: freeing the groupstruck, AI Impacts, September 26.
- ^
Kokotajlo, Daniel (2020) What are the most plausible ‘AI Safety warning shot’ scenarios?, AI Alignment Forum, March 26.
- ^
McCluskey, Peter (2021) AI fire alarm scenarios, Bayesian Investor, December 23.
I agree with that characterisation.
Fwiw I’ve also heard the term “warning sign” being used for non-catastrophes that increase such awareness and concern. In those contexts, “warning shot” has been defined more narrowly as catastrophes that increase such awareness and concern.
Thanks, I reworded the paragraph to reflect this. If you have any references discussing “warning signs”, please let me know. (I recall ASB had written some stuff on the topic but it doesn’t seem to have been published.)
Sorry, no. Yes, I also recall that. I also had some conversations with Eric Drexler where I think this terminology may have been used.