TL;DR: Someone should probably write a grant to produce a spreadsheet/dataset of past instances where people claimed a new technology would lead to societal catastrophe, with variables such as “multiple people working on the tech believed it was dangerous.”
Slightly longer TL;DR: Some AI risk skeptics are mocking people who believe AI could threaten humanity’s existence, saying that many people in the past predicted doom from some new tech. There is seemingly no dataset which lists and evaluates such past instances of “tech doomers.” It seems somewhat ridiculous* to me that nobody has grant-funded a researcher to put together a dataset with variables such as “multiple people working on the technology thought it could be very bad for society.”
*Low confidence: could totally change my mind
———
I have asked multiple people in the AI safety space if they were aware of any kind of “dataset for past predictions of doom (from new technology)”, but have not encountered such a project. There have been some articles and arguments floating around recently such as “Tech Panics, Generative AI, and the Need for Regulatory Caution”, in which skeptics say we shouldn’t worry about AI x-risk because there are many past cases where people in society made overblown claims that some new technology (e.g., bicycles, electricity) would be disastrous for society.
While I think it’s right to consider the “outside view” on these kinds of things, I think that most of these claims 1) ignore examples of where there were legitimate reasons to fear the technology (e.g., nuclear weapons, maybe synthetic biology?), and 2) imply the current worries about AI are about as baseless as claims like “electricity will destroy society,” whereas I would argue that the claim “AI x-risk is >1%” stands up quite well against most current scrutiny.
(These claims also ignore the anthropic argument/survivor bias—that if they ever were right about doom we wouldn’t be around to observe it—but this is less important.)
I especially would like to see a dataset that tracks things like “were the people warning of the risks also the people who were building the technology?” More generally, some measurement of “analytical rigor” also seems really important, e.g., “could the claims have stood up to an ounce of contemporary scrutiny (i.e., without the benefit of hindsight)?”
Absolutely seems worth spending up to $20K to hire researchers to produce such a spreadsheet within the next two-ish months… this could be a critical time period, where people are more receptive to new arguments/responses…?
Just saw this now, after following a link to another comment.
You have almost given me an idea for a research project. I would run the research honestly and report the facts, but my in-going guess is that survivor bias is a massive factor, contrary to what you say here. And that in most cases, the people who believed it could lead to catastrophe were probably right to be concerned. A lot of people have the Y2K bug mentality, in which they didn’t see any disaster and so concluded that it was all a false-alarm, rather than the reality which is that a lot of people did great work to prevent it.
If I look at the different x-risk scenarios the public is most aware of:
Nuclear annihilation—this is very real. As is nuclear winter.
Climate change. This is almost the poster-child for deniers, but in fact there is as yet no reason to believe that the doom-saying predictions are wrong. Everything is going more or less as the scientists predicted, if anything, it’s worse. We have just underestimated the human capacity to stick our heads in the ground and ignore reality*.
Pandemic. Some people see covid as proof that pandemics are not that bad. But we know that, for all the harm it wrought, covid was far from the worst-case. A bioweapon or a natural pandemic.
AI—the risks are very real. We may be lucky with how it evolves, but if we’re not, it will be the machines who are around to write about what happened (and they will write that it wasn’t that bad …)
Etc.
My unique (for this group) perspective on this is that I’ve worked for years on industrial safety, and I know that there are factories out there which have operated for years without a serious safety incident or accident—and someone working in one of those could reach the conclusion that the risks were exaggerated, while being unaware of cases where entire factories or oil-rigs or nuclear power plants have exploded and caused terrible damage and loss of life.
Before I seriously start working on this (in the event that I find time), could you let me know if you’ve since discovered such a data-base?
*We humans are naturally very good at this, because we all know we’re going to die, and we live our lives trying not to think about this fact or desperately trying to convince ourselves of the existence of some kind of afterlife.
Everything is going more or less as the scientists predicted, if anything, it’s worse.
I’m not that focused on climate science, but my understanding is that this is a bit misleading in your context—that there were some scientists in the (90s/2000s?) who forecasted doom or at least major disaster within a few decades due to feedback loops or other dynamics which never materialized. More broadly, my understanding is that forecasting climate has proven very difficult, even if some broad conclusions (e.g., “the climate is changing,” “humans contribute to climate change”) have held up. Additionally, it seems that many engineers/scientists underestimated the pace of alternative energy technology (e.g., solar).
That aside, I would be excited to see someone work on this project, and I still have not discovered any such database.
I’m not sure. IMHO a major disaster is happening with the climate. Essentially, people have a false belief that there is some kind of set-point, and that after a while the temperature will return to that, but this isn’t the case. Venus is an extreme example of an Earth-like planet with a very different climate. There is nothing in physics or chemistry that says Earth’s temperature could not one day exceed 100 C.
It’s always interesting to ask people how high they think sea-level might rise if all the ice melted. This is an uncontroversial calculation which involves no modelling—just looking at how much ice there is, and how much sea-surface area there is. People tend to think it would be maybe a couple of metres. It would actually be 60 m (200 feet). That will take time, but very little time on a cosmic scale, maybe a couple of thousand years.
Right now, if anything what we’re seeing is worse than the average prediction. The glaciers and ice sheets are melting faster. The temperature is increasing faster. Etc. Feedback loops are starting to be powerful. There’s a real chance that the Gulf Stream will stop or reverse, which would be a disaster for Europe, ironically freezing us as a result of global warming …
Among serious climate scientists, the feeling of doom is palpable. I wouldn’t say they are exaggerating. But we, as a global society, have decided that we’d rather have our oil and gas and steaks than prevent the climate disaster. The US seems likely to elect a president who makes it a point of honour to support climate-damaging technologies, just to piss off the scientists and liberals.
Venus is an extreme example of an Earth-like planet with a very different climate. There is nothing in physics or chemistry that says Earth’s temperature could not one day exceed 100 C. [...] [Regarding ice melting -- ] That will take time, but very little time on a cosmic scale, maybe a couple of thousand years.
I’ll be blunt, remarks like these undermine your credibility. But regardless, I just don’t have any experience or contributions to make on climate change, other than re-emphasizing my general impression that, as a person who cares a lot about existential risk and has talked to various other people who also care a lot about existential risk, there seems to be very strong scientific evidence suggesting that extinction is unlikely.
But as a scientist, I feel it’s valuable to speak the truth sometimes, to put my personal credibility on the line in service of the greater good. Venus is an Earth-sized planet which is 400C warmer than Earth, and only a tiny fraction of this is due to it being closer to the sun. The majority is about the % of the sun’s heat that it absorbs vs. reflects. It is an extreme case of global warming. I’m not saying that Earth can be like Venus anytime soon, I’m saying that we have the illusion that Earth has a natural, “stable” temperature, and while it might vary, eventually we’ll return to that temperature. But there is absolutely no scientific or empirical evidence for this.
Earth’s temperature is like a ball balanced in a shallow groove on the top of a steep hill. We’ve never experienced anything outside the narrow groove, so we imagine that it is impossible. But we’ve also never dramatically changed the atmosphere the way we’re doing now. There is, like I said, no fundamental reason why global-warming could not go totally out of control, way beyond 1.5C or 3C or even 20C.
I have struggled to explain this concept, even to very educated, open-minded people who fundamentally agree with my concerns about climate change. So I don’t expect many people to believe me. But intellectually, I want to be honest.
I think it is valuable to keep trying to explain this, even knowing the low probability of success, because right now, statements like “1.5C temperature increase” are just not having the impact of changing people’s habits. And if we do cross a tipping point, it will be too late to start realising this.
TL;DR: Someone should probably write a grant to produce a spreadsheet/dataset of past instances where people claimed a new technology would lead to societal catastrophe, with variables such as “multiple people working on the tech believed it was dangerous.”
Slightly longer TL;DR: Some AI risk skeptics are mocking people who believe AI could threaten humanity’s existence, saying that many people in the past predicted doom from some new tech. There is seemingly no dataset which lists and evaluates such past instances of “tech doomers.” It seems somewhat ridiculous* to me that nobody has grant-funded a researcher to put together a dataset with variables such as “multiple people working on the technology thought it could be very bad for society.”
*Low confidence: could totally change my mind
———
I have asked multiple people in the AI safety space if they were aware of any kind of “dataset for past predictions of doom (from new technology)”, but have not encountered such a project. There have been some articles and arguments floating around recently such as “Tech Panics, Generative AI, and the Need for Regulatory Caution”, in which skeptics say we shouldn’t worry about AI x-risk because there are many past cases where people in society made overblown claims that some new technology (e.g., bicycles, electricity) would be disastrous for society.
While I think it’s right to consider the “outside view” on these kinds of things, I think that most of these claims 1) ignore examples of where there were legitimate reasons to fear the technology (e.g., nuclear weapons, maybe synthetic biology?), and 2) imply the current worries about AI are about as baseless as claims like “electricity will destroy society,” whereas I would argue that the claim “AI x-risk is >1%” stands up quite well against most current scrutiny.
(These claims also ignore the anthropic argument/survivor bias—that if they ever were right about doom we wouldn’t be around to observe it—but this is less important.)
I especially would like to see a dataset that tracks things like “were the people warning of the risks also the people who were building the technology?” More generally, some measurement of “analytical rigor” also seems really important, e.g., “could the claims have stood up to an ounce of contemporary scrutiny (i.e., without the benefit of hindsight)?”
Absolutely seems worth spending up to $20K to hire researchers to produce such a spreadsheet within the next two-ish months… this could be a critical time period, where people are more receptive to new arguments/responses…?
Just saw this now, after following a link to another comment.
You have almost given me an idea for a research project. I would run the research honestly and report the facts, but my in-going guess is that survivor bias is a massive factor, contrary to what you say here. And that in most cases, the people who believed it could lead to catastrophe were probably right to be concerned. A lot of people have the Y2K bug mentality, in which they didn’t see any disaster and so concluded that it was all a false-alarm, rather than the reality which is that a lot of people did great work to prevent it.
If I look at the different x-risk scenarios the public is most aware of:
Nuclear annihilation—this is very real. As is nuclear winter.
Climate change. This is almost the poster-child for deniers, but in fact there is as yet no reason to believe that the doom-saying predictions are wrong. Everything is going more or less as the scientists predicted, if anything, it’s worse. We have just underestimated the human capacity to stick our heads in the ground and ignore reality*.
Pandemic. Some people see covid as proof that pandemics are not that bad. But we know that, for all the harm it wrought, covid was far from the worst-case. A bioweapon or a natural pandemic.
AI—the risks are very real. We may be lucky with how it evolves, but if we’re not, it will be the machines who are around to write about what happened (and they will write that it wasn’t that bad …)
Etc.
My unique (for this group) perspective on this is that I’ve worked for years on industrial safety, and I know that there are factories out there which have operated for years without a serious safety incident or accident—and someone working in one of those could reach the conclusion that the risks were exaggerated, while being unaware of cases where entire factories or oil-rigs or nuclear power plants have exploded and caused terrible damage and loss of life.
Before I seriously start working on this (in the event that I find time), could you let me know if you’ve since discovered such a data-base?
*We humans are naturally very good at this, because we all know we’re going to die, and we live our lives trying not to think about this fact or desperately trying to convince ourselves of the existence of some kind of afterlife.
I’m not that focused on climate science, but my understanding is that this is a bit misleading in your context—that there were some scientists in the (90s/2000s?) who forecasted doom or at least major disaster within a few decades due to feedback loops or other dynamics which never materialized. More broadly, my understanding is that forecasting climate has proven very difficult, even if some broad conclusions (e.g., “the climate is changing,” “humans contribute to climate change”) have held up. Additionally, it seems that many engineers/scientists underestimated the pace of alternative energy technology (e.g., solar).
That aside, I would be excited to see someone work on this project, and I still have not discovered any such database.
I’m not sure. IMHO a major disaster is happening with the climate. Essentially, people have a false belief that there is some kind of set-point, and that after a while the temperature will return to that, but this isn’t the case. Venus is an extreme example of an Earth-like planet with a very different climate. There is nothing in physics or chemistry that says Earth’s temperature could not one day exceed 100 C.
It’s always interesting to ask people how high they think sea-level might rise if all the ice melted. This is an uncontroversial calculation which involves no modelling—just looking at how much ice there is, and how much sea-surface area there is. People tend to think it would be maybe a couple of metres. It would actually be 60 m (200 feet). That will take time, but very little time on a cosmic scale, maybe a couple of thousand years.
Right now, if anything what we’re seeing is worse than the average prediction. The glaciers and ice sheets are melting faster. The temperature is increasing faster. Etc. Feedback loops are starting to be powerful. There’s a real chance that the Gulf Stream will stop or reverse, which would be a disaster for Europe, ironically freezing us as a result of global warming …
Among serious climate scientists, the feeling of doom is palpable. I wouldn’t say they are exaggerating. But we, as a global society, have decided that we’d rather have our oil and gas and steaks than prevent the climate disaster. The US seems likely to elect a president who makes it a point of honour to support climate-damaging technologies, just to piss off the scientists and liberals.
I’ll be blunt, remarks like these undermine your credibility. But regardless, I just don’t have any experience or contributions to make on climate change, other than re-emphasizing my general impression that, as a person who cares a lot about existential risk and has talked to various other people who also care a lot about existential risk, there seems to be very strong scientific evidence suggesting that extinction is unlikely.
I know. :(
But as a scientist, I feel it’s valuable to speak the truth sometimes, to put my personal credibility on the line in service of the greater good. Venus is an Earth-sized planet which is 400C warmer than Earth, and only a tiny fraction of this is due to it being closer to the sun. The majority is about the % of the sun’s heat that it absorbs vs. reflects. It is an extreme case of global warming. I’m not saying that Earth can be like Venus anytime soon, I’m saying that we have the illusion that Earth has a natural, “stable” temperature, and while it might vary, eventually we’ll return to that temperature. But there is absolutely no scientific or empirical evidence for this.
Earth’s temperature is like a ball balanced in a shallow groove on the top of a steep hill. We’ve never experienced anything outside the narrow groove, so we imagine that it is impossible. But we’ve also never dramatically changed the atmosphere the way we’re doing now. There is, like I said, no fundamental reason why global-warming could not go totally out of control, way beyond 1.5C or 3C or even 20C.
I have struggled to explain this concept, even to very educated, open-minded people who fundamentally agree with my concerns about climate change. So I don’t expect many people to believe me. But intellectually, I want to be honest.
I think it is valuable to keep trying to explain this, even knowing the low probability of success, because right now, statements like “1.5C temperature increase” are just not having the impact of changing people’s habits. And if we do cross a tipping point, it will be too late to start realising this.