How did Nick Bostrom come up with the “Simulation argument”*?
Below is an answer Bostrom gave in 2008. (Though note, Pablo shares a comment below that Bostrom might be misremembering this, and he may have taken the idea from Hans Moravec.)
“In my doctoral work, I had studied so-called self-locating beliefs and developed the first mathematical theory of observation selection effects, which affects such beliefs. I had also for many years been thinking a lot about future technological capabilities and their possible impacts on humanity. If one combines these two areas – observation selection theory and the study of future technological capacities – then the simulation argument is only one small inferential step away.
Before the idea was developed in its final form, I had for a couple of years been running a rudimentary version of it past colleagues at coffee breaks during conferences. Typically, the response would be “yeah, that is kind of interesting” and then the conversation would drift to other topics without anything having been resolved.
I was on my way to the gym one evening and was again pondering the argument when it dawned on me that it was more than just coffee-break material and that it could be developed in a more rigorous form. By the time I had finished the physical workout, I had also worked out the essential structure of the argument (which is actually very simple). I went to my office and wrote it up.
(Are there any lessons in this? That new ideas often spring from the combining of two different areas or cognitive structures, which one has previously mastered at sufficiently a deep level, is a commonplace. But an additional possible moral, which may not be as widely appreciated, is that even when we do vaguely realize something, the breakthrough often eludes us because we fail to take the idea seriously enough.)”
Context for this post:
I’m doing some research on “A History of Robot Rights Research,” which includes digging into some early transhumanist / proto-EA type content. I stumbled across this.
I tend to think of researchers as contributing either more through being detail oriented—digging into sources or generating new empirical data—or being really inventive and creative. I definitely fall into the former camp, and am often amazed/confused by the process of how people in the latter camp do what they do. Having found this example, it seemed worth sharing quickly.
*Definition of the simulation argument: “The simulation argument was set forth in a paper published in 2003. A draft of that paper had previously been circulated for a couple of years. The argument shows that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed. The argument has attracted a considerable amount of attention, among scientists and philosophers as well as in the media.”
Note that Hans Moravec, an Austrian-born roboticist, came up with essentially the same idea back in the 1990s. Bostrom was very familiar with Moravec’s work, so it’s likely he encountered it prior to 2003, but then forgot it by the time he made his rediscovery.
“Cryptomnesia occurs when a forgotten memory returns without its being recognized as such by the subject, who believes it is something new and original. It is a memory bias whereby a person may falsely recall generating a thought, an idea, a tune, a name, or a joke,[1] not deliberately engaging in plagiarism but rather experiencing a memory as if it were a new inspiration.”
I haven’t read Moravec’s book very thoroughly, but I ctrl+f’d for “simulation” and couldn’t see anything very explicitly discussing the idea that we might be living in a simulation. There are a number of instances where Moravec talks about running very detailed simulations (and implying that these would be functionally similar to humans). It’s possible (quite likely?) Bostrom didn’t ever see the 1995 article where Moravec “shrugs and waves his hand as if the idea is too obvious.”
Either way, it seems true that (1) the idea itself predates Bostrom’s discussion in his 2003 article, (2) Bostrom’s discussion of this specific idea is more detailed than Moravec’s.
Bostrom (2003) cited Moravec (1988), but not for this specific idea—it’s only for the idea that “One estimate, based on how computationally expensive it is to replicate the functionality of a piece of nervous tissue that we have already understood and whose functionality has been replicated in silico, contrast enhancement in the retina, yields a figure of ~10^14 operations per second for the entire human brain.”
But yeah, his answer to the question “How did you come up with this?” in the 2008 article I linked to in the original post seems misleading, because he doesn’t mention Moravec at all and implies that he came up with the idea himself.
Oh, nice, thanks very much for sharing that. I’ve cited Moravec in the same research report that led me to the Bostrom link I just shared, but hadn’t seen that article and didn’t read Mind Children fully enough to catch that particular idea.
How did Nick Bostrom come up with the “Simulation argument”*?
Below is an answer Bostrom gave in 2008. (Though note, Pablo shares a comment below that Bostrom might be misremembering this, and he may have taken the idea from Hans Moravec.)
“In my doctoral work, I had studied so-called self-locating beliefs and developed the first mathematical theory of observation selection effects, which affects such beliefs. I had also for many years been thinking a lot about future technological capabilities and their possible impacts on humanity. If one combines these two areas – observation selection theory and the study of future technological capacities – then the simulation argument is only one small inferential step away.
Before the idea was developed in its final form, I had for a couple of years been running a rudimentary version of it past colleagues at coffee breaks during conferences. Typically, the response would be “yeah, that is kind of interesting” and then the conversation would drift to other topics without anything having been resolved.
I was on my way to the gym one evening and was again pondering the argument when it dawned on me that it was more than just coffee-break material and that it could be developed in a more rigorous form. By the time I had finished the physical workout, I had also worked out the essential structure of the argument (which is actually very simple). I went to my office and wrote it up.
(Are there any lessons in this? That new ideas often spring from the combining of two different areas or cognitive structures, which one has previously mastered at sufficiently a deep level, is a commonplace. But an additional possible moral, which may not be as widely appreciated, is that even when we do vaguely realize something, the breakthrough often eludes us because we fail to take the idea seriously enough.)”
Context for this post:
I’m doing some research on “A History of Robot Rights Research,” which includes digging into some early transhumanist / proto-EA type content. I stumbled across this.
I tend to think of researchers as contributing either more through being detail oriented—digging into sources or generating new empirical data—or being really inventive and creative. I definitely fall into the former camp, and am often amazed/confused by the process of how people in the latter camp do what they do. Having found this example, it seemed worth sharing quickly.
*Definition of the simulation argument: “The simulation argument was set forth in a paper published in 2003. A draft of that paper had previously been circulated for a couple of years. The argument shows that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed. The argument has attracted a considerable amount of attention, among scientists and philosophers as well as in the media.”
Note that Hans Moravec, an Austrian-born roboticist, came up with essentially the same idea back in the 1990s. Bostrom was very familiar with Moravec’s work, so it’s likely he encountered it prior to 2003, but then forgot it by the time he made his rediscovery.
It’s quite common:
“Cryptomnesia occurs when a forgotten memory returns without its being recognized as such by the subject, who believes it is something new and original. It is a memory bias whereby a person may falsely recall generating a thought, an idea, a tune, a name, or a joke,[1] not deliberately engaging in plagiarism but rather experiencing a memory as if it were a new inspiration.”
https://en.wikipedia.org/wiki/Cryptomnesia
I haven’t read Moravec’s book very thoroughly, but I ctrl+f’d for “simulation” and couldn’t see anything very explicitly discussing the idea that we might be living in a simulation. There are a number of instances where Moravec talks about running very detailed simulations (and implying that these would be functionally similar to humans). It’s possible (quite likely?) Bostrom didn’t ever see the 1995 article where Moravec “shrugs and waves his hand as if the idea is too obvious.”
Either way, it seems true that (1) the idea itself predates Bostrom’s discussion in his 2003 article, (2) Bostrom’s discussion of this specific idea is more detailed than Moravec’s.
Bostrom (2003) cited Moravec (1988), but not for this specific idea—it’s only for the idea that “One estimate, based on how computationally expensive it is to replicate the functionality of a piece of nervous tissue that we have already understood and whose functionality has been replicated in silico, contrast enhancement in the retina, yields a figure of ~10^14 operations per second for the entire human brain.”
But yeah, his answer to the question “How did you come up with this?” in the 2008 article I linked to in the original post seems misleading, because he doesn’t mention Moravec at all and implies that he came up with the idea himself.
Oh, nice, thanks very much for sharing that. I’ve cited Moravec in the same research report that led me to the Bostrom link I just shared, but hadn’t seen that article and didn’t read Mind Children fully enough to catch that particular idea.