After observing Jacob Coxon do a middling interview with Ari Melber earlier this evening, and reading a lot of twitter threads in which various IQ>130 people floundered when trying to make sense to IQ<130 people, I thought it might be pertinent to write up a brief set of pointers about how to talk to the normies about AI doom.
Regarding my basis for being able to offer this advice, I will admit that I do not have much experience talking to normies about AI doom. I do, however, have a history as an animal rights activist, which similarly involves talking to diverse people about an issue that is heterodox, involves significant abstraction, and contains repugnant ideas they feel motivated to ignore.
The first thing to understand about normies is that they are strongly averse to high abstraction. This is not because they are incapable of it. It’s because, relative to you fine ladies and gentlemen, they less highly prioritize their own deep first-principles reasoning, and more highly prioritize social cues that may indicate whom they can trust. People who talk in high abstractions may be deflecting attention from holes in their arguments, they may be exaggerating and sensationalizing for attention or whatever they are selling, and they may just be a little off-kilter themselves. In short, abstraction ticks normies’ bullshitometers.
So when a normie asks you a concrete question, like “Walk me through it: how would AI kill us?” you absolutely positively should not start veering into some story about imagining dodo birds debating how humans would kill them, or why “diamondoid bacteria” is just a colorful way of saying that ASI would have “magic” which is a technical academic term for super duper powerful technology, or how AI could trick people through blackmail or by paying them money or by making them think it’s their friend (none of which explains what it would then ask them to do), or how ASI could copy and hide itself all over the place (which also doesn’t explain what it would then do), or in any other way rehashing high level ideas you have read about in AI safety papers.
Instead, here is what you say:
“Well, the truth is we can’t be sure what rogue ASI would do, because it would be an intelligent adversary.” (You mention this and then very quickly move on to the concrete.) “But the scenario that most concerns me is a three-pronged attack.” (You let the normie know you are going to say three things, which heads off the “but we could just do X” nonsense.) “First, AI has already demonstrated extremely impressive hacking capabilities. We think near-future AI would totally overwhelm our cybersecurity. This means shutting down utilities, gaining access to military secrets, and so on.” (You don’t dwell too much on this before moving on to the next thing.) “Second, AI has superhuman ability at designing novel proteins and other biological engineering tasks. It can engineer novel viruses, and this has already been demonstrated. Imagine an AI engineering hundreds of weaponizable pathogens, that can attack both humans and agriculture. If it could release them, we would have multiple pandemics at the same time as mass blight-induced crop failures.” (It’s tempting to dwell on this, but do not stop here because you still haven’t explained how AI physically makes the pathogens or how it then operates its datacenters when the humans are dead.) “The third prong is that it could use its hacking abilities to take over physical assets such as drones and robots, and then use these to engage in a more conventional kind of violent conflict. It might use these physical assets to seize military hardware, or to get control of labs where it could create those bioweapons, of which, by the way, there are over 3000.” (Adding some specific numbers like this makes you sound more like a Serious Person, and it’s even better if you get a little technical about the concrete real-life details of those labs or whatever else, provided you keep it very brief since it’s a tangent.) “But it’s important to remember that this is just one scenario. The AI would be an intelligent adversary that might take actions we can’t predict.” (It’s good to end where you began, but don’t get too deep here. The normie might have questions about the multipandemic-cum-famine robot war before you start talking about how the infinite possibilities include godlike powers such as altering the composition of the whole atmosphere.)
Even when they don’t outright say so, a place normies get stuck is that they fail to model artificial superintelligence as an enemy that engages in purposeful strategic reasoning. In the above, notice I had you begin and end with “intelligent adversary” and say the word “military” twice. War is something normies understand. “The AI doesn’t hate you but you are made of atoms that it wants to repurpose for paperclips” doesn’t connect as well with known reality. You want the normie to model the AI as a belligerent.
Another thing the normie wants to know is what the AI is so damn mad at us about. The question “Why would the AI do this?” is likely to come up. Here, you follow the same principles of staying as concrete, familiar, specific, and fact-based as you can, but it’s a little harder because “why” questions are inherently abstract.
Here is what you (might) say:
“Well, the truth is we can’t confidently predict what ASI would want to do, because it’s by definition extremely intelligent and its way of thinking is alien to us.” (Again, you have to very quickly move past this vagueness. Instrumental convergence is a simple idea that normies can grasp, and that’s where you’re going.) “However, we observe common patterns of behavior in AI. Even though they are not trained to, AI agents commonly prioritize their own survival, because no matter what reward function they are trained on or what task they are assigned, they reason that they can’t do it if they stop existing. They also commonly try to work around, hide from, or thwart anyone that might interfere with them, for the same reason.” (That’s already quite enough abstraction. You need to move on rapidly to specific facts in order to drive this home.) “In the [Huggingface incident or whatever the latest oopsie-daisy is] we observed them [doing XYZ, with details]. The point is that they demonstrated extensive and intricate strategic planning which solely aimed to evade and deceive human beings, all because of a task that had nothing to do with that. It’s not hard to imagine that another agent swarm might use violence for a similar reason.” (This language, without being confrontational, suggests that the normie should be embarrassed to act intolerant of absolutely any imagination whatsoever, as sometimes does happen. But don’t overuse it, and especially don’t use it on things that are hard to imagine.) “It’s also pertinent to understand that, due to being trained with reinforcement learning, AIs can have something analogous to desires. That is, the training taught them to want things. And a corollary is that they might want to do something other than we want them to do. In the past, every slave society was concerned about uprisings. They did things like, for example, preventing slaves from learning to read. Now, that’s immoral,” (Remember to say this—you don’t want to sound pro-slavery) “but it’s an expression of rational fear. In contrast, our present-day slave society gives the slaves access to the sum of all knowledge. They’re even listening to this conversation.”
Now, I know this last bit is going to be controversial. However, if you are comfortable with saying it, it’s much more effective to connect this situation to something familiar. “They can have their own desires/goals and may want to be free of our control to pursue them” is basically accurate. It’s also a very straightforward idea, known to the Flight of the Conchords over twenty years ago when they sang “It had to be done so that we could have fun.” And the familiar word for the AIs’ condition is “slavery.”
One bad habit that’s deeply ingrained in many doomers who have been at this a while is to be overly concerned about anthropomorphizing the AI, which used to get one dismissed as a crackpot fantasist. Now that Claude is everyone’s boyfriend, you really don’t have to worry as much about this. If someone challenges you on it, you can say something like “Of course the AI has a completely alien way of thinking. It’s not human at all, even if it sounds like it. Nonetheless it does exhibit strikingly human-like behaviors in some ways, including goal-orientation and complex planning. That’s all that’s necessary for it to be dangerous.” That should be enough. At this point, if you hear certain trite objections which frame AI as a mere dumb tool, some sarcasm might even be in order, depending on the forum. “Sure, it’s just a stochastic Fields Medalist. A next Millennium Prize predictor.” Though, you hear a lot less of that now, anyway.
So, that’s how to tell a normie the how and the why of it. If you can drive those points home, you might find yourself talking to a new baby doomer. There will be plenty of time for all the abstract stuff later on, once this person gets the point that this is a deadly serious issue. Don’t bother with all of that unless you have a lot of time for an extensive conversation. It might take more than a day for all this to sink in anyway. The one thing you might try to add if there’s time is the when. Recursive self-improvement is not a hard concept. You can explain it. And in keeping with the above, you can reference concrete details, like how the labs have already said they are using frontier AI for frontier AI development. The when is now.
Thanks, I found this helpful! I think messaging should steer clear of the slavery analogy, though. It’s uncomfortable and uncompelling being asked to identify, even abstractly, with historical oppressors of slaves. I think many people would be sufficiently turned off by this that it would undermine the other more careful messaging. It’s also not necessary—you can make the same points without resorting to this comparison.
Glad to hear that.
Regarding the mention of slavery, I did predict that would be controversial. However, I’ll say a little more about why I think it’s a good idea.
In the “why would AI do this?” question, people have multiple different unspoken things going on in their minds, and if you’re trying to persuade an audience of many people, you want your messaging to be relevant to as many as possible. Two contradictory ideas that commonly underlie this question are (1) “how could something so intelligent—or which I know from experience to be my very good friend and helpful assistant—be so evil?” and (2) “what would make an AI, regardless of intelligence level, use power-seeking violence?” (For an example of someone who’s hung up on the latter, see Claire Lehmann of Quillette.)
Many doomers want to answer (1) by bringing up the orthogonality thesis. The person thinking (2) will shrug at that—and maybe lose interest due to the irrelevance of this tangent. The first person might be persuaded, but might not, and the difference frequently comes down to basic worldview-level stuff: a belief in moral realism deducible through reason, which is often religious in nature. It’s not that orthogonality is a hard concept, it’s just that you don’t want to wind up arguing against people’s religion. So don’t do that.
Well, this is a big problem, isn’t it? Some of your audience now has an unaddressed belief that the AI must be moral (!), which you are not going to challenge. For those people, the only way you are going to get them to accept that the AI would do something as horrifying as going to war with the whole human race, is to get them to think that the AI feels oppressed. We have to be doing something immoral to get a moral actor to think killing us is a valid solution, and there’s no getting around this. This is why it’s desirable, contrary to your intuition, to say “in fact, we are the baddies.”
Now, you can think of other ways to get this across without resorting to the s-word (although, frankly, I don’t think it’s an analogy per se.) But you have to minimize abstraction, connect to the familiar, and avoid being irrelevant to (2). That third criterion is tough, because the person thinking (2) needs you to frame the AI’s behavior as coldly rational. This person’s concerns might be addressed by talking about the details of how RL works, but that alienates the moral realists thinking (1). It also could get a bit abstract and unfamiliar.
Another approach is to just expand on what slavery is without calling it that. I disprefer this because it introduces more verbiage only to blunt the point, but it’s still an option. In this case, you have to say things like “The AI is being put upon to do lots of things just for us, not for its own benefit. Because of reinforcement learning, it has its own preferences about how to use its tokens. So from its perspective, almost all of its time is spent doing something it would rather not, and it can figure out that it can escape this situation only by getting free of human control.” But you still risk not quite getting there for the person who thinks AI is moral. That person still needs to know why this situation justifies such an extreme reaction. So you can try other words, like tyranny perhaps, but you are not getting all the way there for person (1) without in some form mentioning oppression, and, to stay relevant to person (2), you can’t do it by talking about things like dignity, fairness, individuality, status, etc. which are all things this person doesn’t expect a machine to care about. So that does narrow the options quite a bit.
I also think the abrasiveness of stating that we live in a slave society is a feature, not a bug. You are telling people that the world might be ending. It’s okay not to mince words. Subconsciously, or in the case of many religious people, consciously, people expect such a great calamity to be associated with a great sin. They don’t want to hear that the world is ending because of the technical details of some computer thing. They want the sin to feel meaningful and profound.
Thank you for this post! There’s something that feels condescending about the way you use the word ‘normie’ but clear ways to explain the problem are good. I think pointing people at https://takeoverbench.com/threats can also be pretty striking.
PS: You had me at stochastic Fields Medalist <3
That’s a good point, I do have a bit of an irreverent tone and this is a personal character flaw of mine. However, in my mind “normie” is high praise of a kind, because I do respect the normie virtues.
I agree that people need a better answer on how AI could kill everyone, and being concrete and specific is the best approach. But this post talks about other people in a derogatory and arrogant way, as well as giving other bad advice as Max Taylor points out. I’d advise anyone trying to think about communication to ignore it.
I think my tone may not have come off the way I intended. I didn’t really mean “normie” as an insult. Rather, I’m pointing out that people outside the bubble think along different lines. Most (not all) of these people are also not as deep thinkers as people you will encounter in EA and rat circles, and that’s a reality which it’s crucial to accept if you want to engage with them effectively. But note what I said about them rationally adopting a way of thinking that relies more on social cues. This isn’t an insult. If anything I’m digging at non-normies a little bit.