I think the most potentially similar false panic is the concern over “grey goo” and molecular nanotech in the early 2000′s.
I mean, take a look at the concerns of the “centre for responsible nanotech”, which talk about “nanotech arms races” and preventing “rogue use of nanotech”, because “grey goo and military nanobots will not respect borders”. They propose severe international restrictions to keep nanotech development to a single international entity, and predict nanotech will arrive in ten years. (this was written circa 2005).
I hope the parallels to current day AI fears are obvious. Some of the people that bought into the hype (like Drexler and Yudkowsky) are now in the AI risk movement, using the exact same language.
In reality, actual efforts to create molecular nanotech stalled out, because the physics and engineering barriers turned out to be utterly, ridiculously difficult, and beyond the reach of available technology. Generally people accept now that, (absent speedup from an AGI), molecular nanotech is decades or even centuries away, if it’s even possible at all. And pretty much nobody believes that “accidental grey goo” is remotely feasible, given the engineering challenges involved.
This doesn’t mean that AGI fears will turn out the same way, of course, just that a similar panic has occurred before and turned out to be fine.
Good comment, but Drexler actually strikes me as both more moderate and more interesting on AI than just “same as Yudkowsky”. He thinks really intelligent AIs probably won’t be agents with goals at all (at least the first ones we build), and that this means that takeover worries of the Bostrom/Yudkowsky kind are overrated. It’s true that he doesn’t think the risks are zero, but if you look at the section titles of his FHI report, a lot of it is actually devoted to debunking various claims Bostrom/Yudkowksy make in support of the view that takeover risk is high: https://www.fhi.ox.ac.uk/wp-content/uploads/Reframing_Superintelligence_FHI-TR-2019-1.1-1.pdf
I don’t think this effects the point your making, it just seemed a bit unfair on Drexler if I didn’t mention this.
I think the most potentially similar false panic is the concern over “grey goo” and molecular nanotech in the early 2000′s.
I mean, take a look at the concerns of the “centre for responsible nanotech”, which talk about “nanotech arms races” and preventing “rogue use of nanotech”, because “grey goo and military nanobots will not respect borders”. They propose severe international restrictions to keep nanotech development to a single international entity, and predict nanotech will arrive in ten years. (this was written circa 2005).
I hope the parallels to current day AI fears are obvious. Some of the people that bought into the hype (like Drexler and Yudkowsky) are now in the AI risk movement, using the exact same language.
In reality, actual efforts to create molecular nanotech stalled out, because the physics and engineering barriers turned out to be utterly, ridiculously difficult, and beyond the reach of available technology. Generally people accept now that, (absent speedup from an AGI), molecular nanotech is decades or even centuries away, if it’s even possible at all. And pretty much nobody believes that “accidental grey goo” is remotely feasible, given the engineering challenges involved.
This doesn’t mean that AGI fears will turn out the same way, of course, just that a similar panic has occurred before and turned out to be fine.
Good comment, but Drexler actually strikes me as both more moderate and more interesting on AI than just “same as Yudkowsky”. He thinks really intelligent AIs probably won’t be agents with goals at all (at least the first ones we build), and that this means that takeover worries of the Bostrom/Yudkowsky kind are overrated. It’s true that he doesn’t think the risks are zero, but if you look at the section titles of his FHI report, a lot of it is actually devoted to debunking various claims Bostrom/Yudkowksy make in support of the view that takeover risk is high: https://www.fhi.ox.ac.uk/wp-content/uploads/Reframing_Superintelligence_FHI-TR-2019-1.1-1.pdf
I don’t think this effects the point your making, it just seemed a bit unfair on Drexler if I didn’t mention this.