(not well thought-out musings. I’ve only spent a few minutes thinking about this.)
In thinking about the focus on AI within the EA community, the Fermi paradox popped into my head. For anyone unfamiliar with it and who doesn’t want to click through to Wikipedia, my quick summary of the Fermi paradox is basically: if there is such a high probability of extraterrestrial life, why haven’t we seen any indications of it?
On a very naïve level, AI doomerism suggests a simple solution to the Fermi paradox: we don’t see signs of extraterrestrial life because civilizations tend to create unaligned AI, which destroys them. But I suspect that the AI-relevant variation would actually be something more like this:
We claim that a superintelligent AI is going to be a reality soon (maybe between 5 years and 80 years from now), and in general is a benchmark that any civilization would reach eventually. But if superintelligent AI is a thing that civilizations tend to make, why aren’t we seeing any indications of that in the broader universe? If some extraterrestrial civilization made an aligned AI, wouldn’t we see the results of that in a variety of ways? If some extraterrestrial civilization made an unaligned AI, wouldn’t we see the results of that in a variety of ways?
Like many things, I suppose the details matter immensely. Depending on the morality of the creators, an aligned AI might reach spend resources expanding civilization throughout the galaxy, or it might happily putt along maintaining a globe’s agricultural system. Depending on how an unaligned AI is unaligned, it might be focused on turning the whole universe into paperclips, or it might simply kill its creators to prevent them from enduring suffering. So on a very simplistic level it seems that the claim of “civilizations tend to make AI eventually, and it really is a superintelligent and world-changing technology” is consistent with reality of “we don’t observe any signs of extraterrestrial intelligence.”
(not well thought-out musings. I’ve only spent a few minutes thinking about this.)
In thinking about the focus on AI within the EA community, the Fermi paradox popped into my head. For anyone unfamiliar with it and who doesn’t want to click through to Wikipedia, my quick summary of the Fermi paradox is basically: if there is such a high probability of extraterrestrial life, why haven’t we seen any indications of it?
On a very naïve level, AI doomerism suggests a simple solution to the Fermi paradox: we don’t see signs of extraterrestrial life because civilizations tend to create unaligned AI, which destroys them. But I suspect that the AI-relevant variation would actually be something more like this:
Like many things, I suppose the details matter immensely. Depending on the morality of the creators, an aligned AI might reach spend resources expanding civilization throughout the galaxy, or it might happily putt along maintaining a globe’s agricultural system. Depending on how an unaligned AI is unaligned, it might be focused on turning the whole universe into paperclips, or it might simply kill its creators to prevent them from enduring suffering. So on a very simplistic level it seems that the claim of “civilizations tend to make AI eventually, and it really is a superintelligent and world-changing technology” is consistent with reality of “we don’t observe any signs of extraterrestrial intelligence.”