Mal­ig­nant AI failure mode

TagLast edit: 16 Jun 2021 9:28 UTC by EA Wiki assistant

An AI failure mode is a way in which a project to develop machine superintelligence may fail. Malignant AI failure modes are AI failure modes that result in an existential catastrophe.

Nick Bostrom categorizes malignant AI failure modes into three basic types (Bostrom 2014: 119-126): perverse instantiation, which involves the satisfaction of an AI’s goals in ways contrary to the intentions of those who programmed it; infrastructure profusion, which involves the transformation of large parts of the accessible universe into infrastructure in the service of some goal that impedes the realization of humanity’s longterm potential; and mind crime, which involves the mistreatment of morally relevant computational processes.


Bostrom, Nick (2014) Superintelligence: Paths, Dangers, Strategies, Oxford: Oxford University Press.

Related entries

existential risk | infrastructure profusion | mind crime | perverse instantiation | superintelligence

Against GDP as a met­ric for timelines and take­off speeds

kokotajlod29 Dec 2020 17:50 UTC
41 points
6 comments14 min readEA link
No comments.