I think when trying to get people concerned about “AI safety,” there are a number of “nearby messages” or misunderstandings that are easy for other people to hear, and are potentially very net negative to spread. For example, if you are trying to communicate “AGI systems will one day be very powerful and dangerous, they are by default controllable, therefore we should slow down and try to develop alignment techniques first” people might instead hear:
AGI is powerful, therefore I should work on/fund/invest in it.
AGI is powerful and dangerous, like nuclear weapons. Therefore my country should develop it before other countries.
AGI is powerful and dangerous and uncontrollable, like bioweapons. I’m a safety conscious person, therefore my country/company should develop AGI first.
AGI like ChatGPT and GPT-4 is powerful and dangerous and uncontrollable. Therefore we need to put a stop to ChatGPT.
This is a bad message to spread because, among other issues, we might have a “crying wolf” problem.
I’m not saying these issues in communication are insurmountable. But I think they’re pretty serious, and have arguably historically led to a lot of x-risk. So I think we should not be cavalier in communications about AI safety. The strongest counterargument I’m aware of is something like “This time is different” with the idea that AI progress is already so much in the news that capabilities externalities are minimal. But I think this is not currently the world we live in, and I will be pretty worried about spreading messages that “AI is powerful” or even “AI is powerful and dangerous” to national security folks, or political actors more generally.
I think the best case scenario is convincing people who already believe “AGI is powerful” that AGI is dangerous, rather than messaging on both issues at once. To that end, I think prematurely chasing AI safety messages is bad.
Not to be rude but this seems like a lot of worrying about nothing. “AI is powerful and uncontrollable and could kill all of humanity, like seriously” is not a complicated message. I’m actually quite scared if AI Safety people are hesitant to communicate because they think the misinterpretation will be as bad as you are saying here; this is a really strong assumption, an untested one at that,and the opportunity cost of not pursuing media coverage is enormous.
The primary purpose of media coverage is to introduce the problem, not to immediately push for the solution. I stated ways that different actors taking the problem more seriously would lead to progress; I’m not sure that a delay is actually the main impact. On this last point, note that (as I expected when it was first released) the main effect of the FLI letter is that a lot more people have heard of AI Safety and people who have heard of it are taking it more seriously (the latter based largely on Twitter observations), not that a delay is actually being considered.
I don’t actually know where you’re getting “these issues in communication...historically have led to a lot of x-risk” from. There was no large public discussion about nuclear weapons before initial use (and afterwards we settled into the most reasonable approach there was for preventing nuclear war, namely MAD) or gain-of-function research. The track record of “tell people about problems and they become more concerned about these problems”, on the other hand, is very good.
Not to be rude but this seems like a lot of worrying about nothing. “AI is powerful and uncontrollable and could kill all of humanity, like seriously” is not a complicated message.
To first order, the problem isn’t that the message is complicated. “Bioterrorism might kill you, here are specific viruses that they can use, we should stop that.” is also not a complicated message, but it’ll be a bad idea to indiscriminately spread that message as well.
this is a really strong assumption, an untested one at that
Well there was DeepMind, and then OpenAI, and then Anthropic.
I stated ways that different actors taking the problem more seriously would lead to progress; I’m not sure that a delay is actually the main impact. On this last point, note that (as I expected when it was first released) the main effect of the FLI letter is that a lot more people have heard of AI Safety and people who have heard of it are taking it more seriously (the latter based largely on Twitter observations), not that a delay is actually being considered.
I don’t view this as a crux. I weakly think additional attention is a cost, not a benefit.
I don’t actually know where you’re getting “these issues in communication...historically have led to a lot of x-risk” from
I meant in AI. Also I feel like this might be the crux here. I currently think that past communications (like early Yudkowsky and Superintelligence) have done a lot of harm (though there might have been nontrivial upsides as well). If you don’t believe this you should be more optimistic about indiscriminate AI safety comms than I am, though maybe not to quite the same extent as the OP.
Tbh in contrast with the three target groups you mentioned, I feel more generally optimistic about the “public’s” involvement. I can definitely see worlds where mass outreach is net positive, though of course this is a sharp departure from past attempts (and failures) in communication.
Ahh, I didn’t read it as you talking about the effects of Eliezer’s past outreach. I strongly buy “this time is different”, and not just because of the salience of AI in tech. The type of media coverage we’re getting is very different: the former CEO of Google advocating AI risk and a journalist asking about AI risk in the White House press briefing is just nothing like we’ve ever seen before. We’re reaching different audiences here. The AI landscape is also very different; AI risk arguments are a lot more convincing when we have a very good AI to point to (GPT-4) and when we have facts like “a majority of AI researchers think p(AI killing humanity)>10%”.
But even if you believe this time won’t be different, I think we need to think critically about which world we would rather live in:
the current one, where AI Capabilities research keeps humming along with what seems to be inadequate AI Safety research and nobody outside of EA is really paying attention to AI Safety. All we can do is hope that AI risk isn’t as plausible as Eliezer thinks and that Sam Altman is really careful.
One where there is another SOTA AI capabilities lab, maybe owned by the government, but AI is treated as a dangerous and scary technology that must be treated with care. We have more alignment research, the government keeps tabs on AI labs to make sure they’re not doing anything stupid and maybe adds red tape that slow them down, and AI capabilities researchers everywhere don’t do obviously stupid things.
Let’s even think about the history here. Early Eliezer advocating for AGI to prevent nanotech from killing all of humanity was probably bad. But I am unconvinced that Eliezer’s advocacy from afterwards up until 2015 or whatever was net-negative. My understanding is that though his work led to development of AI capabilities labs, there was nobody at the time working on alignment anyway. This reflex of “AI capabilities research bad” only holds if there is sufficient progress on ensuring AI safety in the meantime.
One last note, on “power”. Assuming Eliezer isn’t horribly wrong about things, the worlds in which we survive AI are those where AI is widely acknowledged as extremely powerful. We’re just not going to make it if policy-makers and/or tech people don’t understand what they are dealing with here. Maybe there are reasons to delay this understanding a few years—I personally strongly oppose this—but let’s be clear about this.
“AI is powerful and uncontrollable and could kill all of humanity, like seriously” is not a complicated message.
Anything longer than 8 morphemes is probably not going to survive Twitter or CNN getting their hands on it. I like the original version (“Literally everyone will die”) better.
I usually agree with @Linch , but strongly disagree here. I struggle to understand the causal pathways for which misunderstanding or “nearby messages” are going to do more harm than good. I also think the 4 thoughts that were bullet pointed are unlikely misunderstandings. And even if people did hear those, it’s good that people have started thinking about it.
More coverage = better, and the accuracy and nuance isn’t so important right now.
I will just copy paste from the OP because they put it so well.
“the public is starting from a place of ~complete ignorance. Anyone reading about AI Safety for the first time is not going to totally absorb the details of the problem. They won’t notice if you e.g. inaccurately describe an alignment approach—they probably won’t remember much that you say beyond “AI could kill us all, like seriously”. And honestly, this is the most important part anyway. A tech person interested in learning the technical details of the problem will seek out the better coverage and find one of the excellent explainers that already exist. A policymaker wanting to regulate this will reach out to experts. You as a communicator just have to spread the message.
I’ve got a friend who often says (kind of jokingly) “We’re all going to die” when he talks about AI. It gets people interested, makes them laugh and gets the word out there.
I usually agree with @Linch , but strongly disagree here. I struggle to understand the causal pathways for which misunderstanding or “nearby messages” are going to do more harm than good.
I think AGI research is bad. I think starting AGI companies is bad. I think funding AGI companies is bad. I think working at AGI companies is bad. I think nationalizing and subsidizing AGI companies is probably bad. I think AGI racing is bad. I think hype that causes the above is bad. I think outreach and community building that causes the above is bad.
Also the empirical track record here is pretty bad.
Agree with this 100% “I think AGI research is bad. I think starting AGI companies is bad. I think funding AGI companies is bad. I think working at AGI companies is bad. I think nationalizing and subsidizing AGI companies is probably bad. I think AGI racing is bad.”
Thanks, are you arguing that raising AI safety awareness will do more harm than good, through increasing the hype and profile of AI? That’s interesting will have to think about it!
The empirical track record is that the top 3 AI research labs (Anthropic, DeepMind, and OpenAI) were all started by people worried that AI would be unsafe, who then went on to design and implement a bunch of unsafe AIs.
100% agree. I’m sometimes confused on this “evidence based” forum as to why this doesn’t get the front page attention and traction.
At a guess, perhaps some people involved in the forum here are friends, or connected to some of the people involved in these orgs and want to avoid confronting this head on? Or maybe they want to keep good relationships with them so they can still influence them to some degree?
I think when trying to get people concerned about “AI safety,” there are a number of “nearby messages” or misunderstandings that are easy for other people to hear, and are potentially very net negative to spread. For example, if you are trying to communicate “AGI systems will one day be very powerful and dangerous, they are by default controllable, therefore we should slow down and try to develop alignment techniques first” people might instead hear:
AGI is powerful, therefore I should work on/fund/invest in it.
AGI is powerful and dangerous, like nuclear weapons. Therefore my country should develop it before other countries.
AGI is powerful and dangerous and uncontrollable, like bioweapons. I’m a safety conscious person, therefore my country/company should develop AGI first.
AGI like ChatGPT and GPT-4 is powerful and dangerous and uncontrollable. Therefore we need to put a stop to ChatGPT.
This is a bad message to spread because, among other issues, we might have a “crying wolf” problem.
I’m not saying these issues in communication are insurmountable. But I think they’re pretty serious, and have arguably historically led to a lot of x-risk. So I think we should not be cavalier in communications about AI safety. The strongest counterargument I’m aware of is something like “This time is different” with the idea that AI progress is already so much in the news that capabilities externalities are minimal. But I think this is not currently the world we live in, and I will be pretty worried about spreading messages that “AI is powerful” or even “AI is powerful and dangerous” to national security folks, or political actors more generally.
I think the best case scenario is convincing people who already believe “AGI is powerful” that AGI is dangerous, rather than messaging on both issues at once. To that end, I think prematurely chasing AI safety messages is bad.
Not to be rude but this seems like a lot of worrying about nothing. “AI is powerful and uncontrollable and could kill all of humanity, like seriously” is not a complicated message. I’m actually quite scared if AI Safety people are hesitant to communicate because they think the misinterpretation will be as bad as you are saying here; this is a really strong assumption, an untested one at that, and the opportunity cost of not pursuing media coverage is enormous.
The primary purpose of media coverage is to introduce the problem, not to immediately push for the solution. I stated ways that different actors taking the problem more seriously would lead to progress; I’m not sure that a delay is actually the main impact. On this last point, note that (as I expected when it was first released) the main effect of the FLI letter is that a lot more people have heard of AI Safety and people who have heard of it are taking it more seriously (the latter based largely on Twitter observations), not that a delay is actually being considered.
I don’t actually know where you’re getting “these issues in communication...historically have led to a lot of x-risk” from. There was no large public discussion about nuclear weapons before initial use (and afterwards we settled into the most reasonable approach there was for preventing nuclear war, namely MAD) or gain-of-function research. The track record of “tell people about problems and they become more concerned about these problems”, on the other hand, is very good.
(also: premature??? really???)
To first order, the problem isn’t that the message is complicated. “Bioterrorism might kill you, here are specific viruses that they can use, we should stop that.” is also not a complicated message, but it’ll be a bad idea to indiscriminately spread that message as well.
Well there was DeepMind, and then OpenAI, and then Anthropic.
I don’t view this as a crux. I weakly think additional attention is a cost, not a benefit.
I meant in AI. Also I feel like this might be the crux here. I currently think that past communications (like early Yudkowsky and Superintelligence) have done a lot of harm (though there might have been nontrivial upsides as well). If you don’t believe this you should be more optimistic about indiscriminate AI safety comms than I am, though maybe not to quite the same extent as the OP.
Tbh in contrast with the three target groups you mentioned, I feel more generally optimistic about the “public’s” involvement. I can definitely see worlds where mass outreach is net positive, though of course this is a sharp departure from past attempts (and failures) in communication.
Ahh, I didn’t read it as you talking about the effects of Eliezer’s past outreach. I strongly buy “this time is different”, and not just because of the salience of AI in tech. The type of media coverage we’re getting is very different: the former CEO of Google advocating AI risk and a journalist asking about AI risk in the White House press briefing is just nothing like we’ve ever seen before. We’re reaching different audiences here. The AI landscape is also very different; AI risk arguments are a lot more convincing when we have a very good AI to point to (GPT-4) and when we have facts like “a majority of AI researchers think p(AI killing humanity)>10%”.
But even if you believe this time won’t be different, I think we need to think critically about which world we would rather live in:
the current one, where AI Capabilities research keeps humming along with what seems to be inadequate AI Safety research and nobody outside of EA is really paying attention to AI Safety. All we can do is hope that AI risk isn’t as plausible as Eliezer thinks and that Sam Altman is really careful.
One where there is another SOTA AI capabilities lab, maybe owned by the government, but AI is treated as a dangerous and scary technology that must be treated with care. We have more alignment research, the government keeps tabs on AI labs to make sure they’re not doing anything stupid and maybe adds red tape that slow them down, and AI capabilities researchers everywhere don’t do obviously stupid things.
Let’s even think about the history here. Early Eliezer advocating for AGI to prevent nanotech from killing all of humanity was probably bad. But I am unconvinced that Eliezer’s advocacy from afterwards up until 2015 or whatever was net-negative. My understanding is that though his work led to development of AI capabilities labs, there was nobody at the time working on alignment anyway. This reflex of “AI capabilities research bad” only holds if there is sufficient progress on ensuring AI safety in the meantime.
One last note, on “power”. Assuming Eliezer isn’t horribly wrong about things, the worlds in which we survive AI are those where AI is widely acknowledged as extremely powerful. We’re just not going to make it if policy-makers and/or tech people don’t understand what they are dealing with here. Maybe there are reasons to delay this understanding a few years—I personally strongly oppose this—but let’s be clear about this.
Anything longer than 8 morphemes is probably not going to survive Twitter or CNN getting their hands on it. I like the original version (“Literally everyone will die”) better.
I usually agree with @Linch , but strongly disagree here. I struggle to understand the causal pathways for which misunderstanding or “nearby messages” are going to do more harm than good. I also think the 4 thoughts that were bullet pointed are unlikely misunderstandings. And even if people did hear those, it’s good that people have started thinking about it.
More coverage = better, and the accuracy and nuance isn’t so important right now.
I will just copy paste from the OP because they put it so well.
“the public is starting from a place of ~complete ignorance. Anyone reading about AI Safety for the first time is not going to totally absorb the details of the problem. They won’t notice if you e.g. inaccurately describe an alignment approach—they probably won’t remember much that you say beyond “AI could kill us all, like seriously”. And honestly, this is the most important part anyway. A tech person interested in learning the technical details of the problem will seek out the better coverage and find one of the excellent explainers that already exist. A policymaker wanting to regulate this will reach out to experts. You as a communicator just have to spread the message.
I’ve got a friend who often says (kind of jokingly) “We’re all going to die” when he talks about AI. It gets people interested, makes them laugh and gets the word out there.
I think AGI research is bad. I think starting AGI companies is bad. I think funding AGI companies is bad. I think working at AGI companies is bad. I think nationalizing and subsidizing AGI companies is probably bad. I think AGI racing is bad. I think hype that causes the above is bad. I think outreach and community building that causes the above is bad.
Also the empirical track record here is pretty bad.
Agree with this 100% “I think AGI research is bad. I think starting AGI companies is bad. I think funding AGI companies is bad. I think working at AGI companies is bad. I think nationalizing and subsidizing AGI companies is probably bad. I think AGI racing is bad.”
Thanks, are you arguing that raising AI safety awareness will do more harm than good, through increasing the hype and profile of AI? That’s interesting will have to think about it!
What do you mean by “the empirical track?”
Thanks!
The empirical track record is that the top 3 AI research labs (Anthropic, DeepMind, and OpenAI) were all started by people worried that AI would be unsafe, who then went on to design and implement a bunch of unsafe AIs.
100% agree. I’m sometimes confused on this “evidence based” forum as to why this doesn’t get the front page attention and traction.
At a guess, perhaps some people involved in the forum here are friends, or connected to some of the people involved in these orgs and want to avoid confronting this head on? Or maybe they want to keep good relationships with them so they can still influence them to some degree?
Should be “empirical track record.” Sorry, fixed.