The median AI researcher estimates even odds of human-level AI between 2035 and 2050 so the prospect that AI is possible and achievable within decades is large enough to worry about.
Predictions about when we will achieve human-level AI have been wildly inaccurate in the past[1]. I don’t think the predictions of current AI researchers is particularly useful data point.
Assuming that we do in fact achieve human-level AI at some point, then if we’re going to avoid Pascal’s Mugging you need to present compelling evidence that the path from human-level AI → superintelligent/singularity/end of humanity level AI is (a) a thing which is likely (i.e. p >> 10^-50), (b) a thing that is likely to be bad for humanity, and (c) that we have a credible chance of altering the outcome in a way that benefits us.
I’ve seen some good arguments for (b), much less so for (a) and (c). Are there good arguments for these, or am I missing some other line of reasoning (very possible) that makes a non-Pascal’s Mugging argument for AI research?
Predictions about when we will achieve human-level AI have been wildly inaccurate in the past[1]. I don’t think the predictions of current AI researchers is particularly useful data point.
I agree that they’re not reliable, but there’s not much better. We’re basically citing the same body of surveys. On a compromise reading I suppose they suggest that AI will likely happen anywhen between this decade and a few centuries, with most of the weight in this one, which sounds right to me.
a) AI is already superhuman in a bunch of domains that are increasingly complex, starting from backgammon and chess to Jeopardy, driving, and image recognition. Computing power is still increasing, though less quickly than before, and in a more parallel direction. Algorithms are also getting better. There’s also a parallel path to superintelligence through brain emulation. So AI getting superhuman in some domains is science fact already.
Once AI gets more intelligent than one human in certain important domains, such as i) information security ii) trading iii) manipulating military hardware, or iv) persuasion, it will have significant power. See Scott’s colourful descriptions. So p < 10^-2 cannot be right here
c) This is harder. The best achievements of the AI community so far are making some interesting theoretical discoveries re cooperation and decision theory (MIRI), attracting millions in donations from eccentric billionaires (FLI), convening dozens of supportive AI experts (FLI), writing a popular book (Bostrom) and meeting with high levels of government in UK (FHI) and Germany (CSER). This is great, though none yet shows the path to friendly AI. There are suggestions for how to make friendly AI. Even if there weren’t, there’d be a nontrivial chance that this emerging x-risk-aware apparatus would find them, given that it it young and quickly gaining momentum. MIRI’s approach would require a lot more technical exploration, while a brain emulation approach would require a heap more resources, as well as progress in hardware and brain-scanning technology. I think this is the substance that has to be engaged with to push this discussion forward, and potentially also to improve AI safety efforts.
While Ryan’s below rebuttal to A sways me a little bit, I have reservations until I learn more about counterarguments to predictions of an intelligence explosion
The biggest bottleneck put here is C, which gets to the heart of why I wouldn’t be ready to focus primarily on A.I. safety research among all causes right now. Thanks to you and Ryan for illustrating my concerns also, ones I can point to in the future when discussing A.I. safety research.
Predictions about when we will achieve human-level AI have been wildly inaccurate in the past[1]. I don’t think the predictions of current AI researchers is particularly useful data point.
Assuming that we do in fact achieve human-level AI at some point, then if we’re going to avoid Pascal’s Mugging you need to present compelling evidence that the path from human-level AI → superintelligent/singularity/end of humanity level AI is (a) a thing which is likely (i.e. p >> 10^-50), (b) a thing that is likely to be bad for humanity, and (c) that we have a credible chance of altering the outcome in a way that benefits us.
I’ve seen some good arguments for (b), much less so for (a) and (c). Are there good arguments for these, or am I missing some other line of reasoning (very possible) that makes a non-Pascal’s Mugging argument for AI research?
[1] https://intelligence.org/files/PredictingAI.pdf
I agree that they’re not reliable, but there’s not much better. We’re basically citing the same body of surveys. On a compromise reading I suppose they suggest that AI will likely happen anywhen between this decade and a few centuries, with most of the weight in this one, which sounds right to me.
a) AI is already superhuman in a bunch of domains that are increasingly complex, starting from backgammon and chess to Jeopardy, driving, and image recognition. Computing power is still increasing, though less quickly than before, and in a more parallel direction. Algorithms are also getting better. There’s also a parallel path to superintelligence through brain emulation. So AI getting superhuman in some domains is science fact already.
Once AI gets more intelligent than one human in certain important domains, such as i) information security ii) trading iii) manipulating military hardware, or iv) persuasion, it will have significant power. See Scott’s colourful descriptions. So p < 10^-2 cannot be right here
c) This is harder. The best achievements of the AI community so far are making some interesting theoretical discoveries re cooperation and decision theory (MIRI), attracting millions in donations from eccentric billionaires (FLI), convening dozens of supportive AI experts (FLI), writing a popular book (Bostrom) and meeting with high levels of government in UK (FHI) and Germany (CSER). This is great, though none yet shows the path to friendly AI. There are suggestions for how to make friendly AI. Even if there weren’t, there’d be a nontrivial chance that this emerging x-risk-aware apparatus would find them, given that it it young and quickly gaining momentum. MIRI’s approach would require a lot more technical exploration, while a brain emulation approach would require a heap more resources, as well as progress in hardware and brain-scanning technology. I think this is the substance that has to be engaged with to push this discussion forward, and potentially also to improve AI safety efforts.
While Ryan’s below rebuttal to A sways me a little bit, I have reservations until I learn more about counterarguments to predictions of an intelligence explosion The biggest bottleneck put here is C, which gets to the heart of why I wouldn’t be ready to focus primarily on A.I. safety research among all causes right now. Thanks to you and Ryan for illustrating my concerns also, ones I can point to in the future when discussing A.I. safety research.