[9] The Future of Human Evolution (Bostrom, 2004)

Link post

Will’s quick summary/​hot takes:

  • The most important sections are 2, 3, and 11

  • Section 2: “Two dystopian ‘upward’ evolutionary scenarios”

    • Both of these evolutionary scenarios seem pretty awful

      • (Section 3 explains why one might think the end state of evolution might be so wild. I’ll get to section 3 in due course.)

      • Scenario one, “The Mindless Outsourcers,” is very similar in spirit to the following passage from “Meditations on Moloch” (Alexander, 2014):

        • “(Moloch whose eyes are a thousand blind windows!) Everything the human race has worked for – all of our technology, all of our civilization, all the hopes we invested in our future – might be accidentally handed over to some kind of unfathomable blind idiot alien god that discards all of them, and consciousness itself, in order to participate in some weird fundamental-level mass-energy economy that leads to it disassembling Earth and everything on it for its component atoms.”

      • Scenario two, “All-Work-And-No-Fun,” is partly similar to what Hanson (2016) describes in The Age of Em:

        • “Ems will work nearly all the time but choose to remember an existence that is nearly all leisure. Some ems will be ‘open-source lovers’; all will be markedly more religious and also swear more often. The em economy will double every month, and competition will drive nearly all wages down to subsistence levels. Surveillance will be total.”

          • (Note: I took the above quote from this Guardian review of The Age of Em.)

  • Section 3: “Ours is an evolutionary disequilibrium”

    • So, why might one think that the end state of evolution will be so different from the current state of the world? Because the current scenario we live in is likely unstable:

      • “Excess resources allow a temporary respite. We live in the dream time.” (Mowshowitz, 2019)

      • “This is a rare confluence of circumstances where we are unusually safe from multipolar traps, and as such weird things like art and science and philosophy and love can flourish.” (Alexander, 2014)

  • Section 11: “Only a singleton could control evolution”

    • This is an important and decision relevant—hopefully—conclusion

    • For a great description of what a singleton is, see “What is a Singleton?” (Bostrom, 2005)

    • My bet for the singleton we should hope for… aligned/​friendly AGI

No comments.