Unfortunately I worry that paperclips might exist at an un-de-throne-able optimum point of memetic sharability vs educationalness.
The great thing about the paperclip meme is that it is an AI safety Koan—it helps people learn about orthogonality, because until you until you understand orthogonality the story sounds totally insane, but then once you make the leap then everything fits perfectly. (Similarly, IMO Koans are supposed to be stories that make a kind of sense to a spiritually-attained person, but sound ridiculous to anyone else, thus helping people become spiritually-attained.)
By contrast, other more complex or realistic stories (like an AI that makes everyone happier and happier and then starts wireheading them, or about an unsafe AI that is created as part of a great-power military arms race), hew closer to existing Hollywood stories about evil robots, mad scientists, and genies granting tricky wishes. The very plausibility of this story detracts from the Koan-like ability to teach people “AI safety isn’t about fear of evil overlords or about wrongdoers getting their comeuppance, it is literally a technical problem”.
IMO, instead of trying to dethrone paperclips as the snappiest and most memetic story, what the AI safety movement could use is more immersive, detailed, and well-written stories basically all along the pareto frontier of memability vs realism that we can point people to as a next step, after they’ve heard about paperclips. Paul Christiano’s detailed future-histories of AI slowly derailing humanity’s future might exist at the far end of realism, but it’s long and very sophisticated… but it would be sweet to have some more-memeable, medium-realistic, medium-length, well-produced tales in between that and “paperclips”. (“Paperclips” translated well into videogame form… maybe a more sophisticated AI story could also be told as a simple game, to encourage the systems thinking vs associations of hollywood tales?)
New EA Cause Area: “dethrone paperclips”!
Unfortunately I worry that paperclips might exist at an un-de-throne-able optimum point of memetic sharability vs educationalness.
The great thing about the paperclip meme is that it is an AI safety Koan—it helps people learn about orthogonality, because until you until you understand orthogonality the story sounds totally insane, but then once you make the leap then everything fits perfectly. (Similarly, IMO Koans are supposed to be stories that make a kind of sense to a spiritually-attained person, but sound ridiculous to anyone else, thus helping people become spiritually-attained.)
By contrast, other more complex or realistic stories (like an AI that makes everyone happier and happier and then starts wireheading them, or about an unsafe AI that is created as part of a great-power military arms race), hew closer to existing Hollywood stories about evil robots, mad scientists, and genies granting tricky wishes. The very plausibility of this story detracts from the Koan-like ability to teach people “AI safety isn’t about fear of evil overlords or about wrongdoers getting their comeuppance, it is literally a technical problem”.
IMO, instead of trying to dethrone paperclips as the snappiest and most memetic story, what the AI safety movement could use is more immersive, detailed, and well-written stories basically all along the pareto frontier of memability vs realism that we can point people to as a next step, after they’ve heard about paperclips. Paul Christiano’s detailed future-histories of AI slowly derailing humanity’s future might exist at the far end of realism, but it’s long and very sophisticated… but it would be sweet to have some more-memeable, medium-realistic, medium-length, well-produced tales in between that and “paperclips”. (“Paperclips” translated well into videogame form… maybe a more sophisticated AI story could also be told as a simple game, to encourage the systems thinking vs associations of hollywood tales?)