Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

On June 22nd, there was a “Munk Debate”, facilitated by the Canadian Aurea Foundation, on the question whether “AI research and development poses an existential threat” (you can watch it here, which I highly recommend). On stage were Yoshua Bengio and Max Tegmark as proponents and Yann LeCun and Melanie Mitchell as opponents of the central thesis. This seems like an excellent opportunity to compare their arguments and the effects they had on the audience, in particular because in the Munk Debate format, the audience gets to vote on the issue before and after the debate.

The vote at the beginning revealed 67% of the audience being pro the existential threat hypothesis and 33% against it. Interestingly, it was also asked if the listeners were prepared to change their minds depending on how the debate went, which 92% answered with “yes”. The moderator later called this extraordinary and a possible record for the format. While this is of course not representative for the general public, it mirrors the high uncertainty that most ordinary people feel about AI and its impacts on our future.

I am of course heavily biased. I would have counted myself among the 8% of people who were unwilling to change their minds, and indeed I’m still convinced that we need to take existential risks from AI very seriously. While Bengio and Tegmark have strong arguments from years of alignment research on their side, LeCun and Mitchell have often made weak claims in public. So I was convinced that Bengio and Tegmark would easily win the debate.

However, when I skipped to the end of the video before watching it, there was an unpleasant surprise waiting for me: at the end of the debate, the audience had seemingly switched to a more skeptical view, with now only 61% accepting an existential threat from AI and 39% dismissing it.

What went wrong? Had Max Tegmark and Yoshua Bengio really lost a debate against two people I hadn’t taken very seriously before? Had the whole debate somehow been biased against them?

As it turned out, things were not so clear. At the end, the voting system apparently broke down, so the audience wasn’t able to vote on the spot. Instead, they were later asked for their vote by email. It is unknown how many people responded, so the difference can well be a random error. However, it does seem to me that LeCun and Mitchell, although clearly having far weaker arguments, came across quite convincing. A simple count of the hands of the people behind the stage, who can be seen in the video, during a hand vote results almost in a tie. The words of the moderator also seem to indicate that he couldn’t see a clear majority for one side in the audience, so the actual shift may have been even worse.

In the following, I assume that Bengio and Tegmark were indeed not as convincing as I had hoped. It seems worthwhile to look at this in some more detail to learn from it for future discussions.

I will not give a detailed description of the debate; I recommend you watch it yourself. However, I will summarize some key points and will give my own opinion on why this may have gone badly from an AI safety perspective, as well as some learnings I extracted for my own outreach work.

The debate was structured in a good way and very professionally moderated by Munk Debate’s chair Rudyard Griffiths. If anything, he seemed to be supportive of an existential threat from AI; he definitely wasn’t biased against it. At the beginning, each participant gave a 6-minute opening statement, then each one could reply to what the others had said in a brief rebuttal. After that, there was an open discussion for about 40 minutes, until the participants could again summarize their viewpoints in a closing statement. Overall, I would say the debate was fair and no side made significant mistakes or blunders.

I will not repeat all the points the participants made, but give a brief overview of their stance on various issues as I understood them in the following table:

TegmarkBengioLeCunMitchell
Is AI R&D an existential risk?YesYesNo, we will keep AI under controlNo, this is just science fiction/​ not grounded in science
What is the probability of an existential risk from AI?High enough to be concerned (>10%)Edit: probability of ASI 10%-50% according to people I spoke toAs small as that of being wiped out by an asteroidNegligible
Is ASI possible in the foreseeable future?YesYes, 5-20 yearsYes, although there are still important elements missingNo
Is there an x-risk from malicious actors using AI?YesYesNo, because the good guys will have superior AINo, AI will not make already existing threats much worse
Is there an x-risk from rogue AI?YesYesNo, because we won’t build AI that isn’t safeNo, AI will be subhuman for a long time
Is there an x-risk from human dis-empower-ment?YesYesNo, AI will always be docile and make people strongerNo, AI will be subhuman for a long time
Will ASI seek power?YesYesNo, we will make AI docile, intel-ligence is not correlated with dominanceNo, AI has no will of its own
Is the orthogonality thesis correct? (The term wasn’t mentioned directly in the debate)YesYesNo, intelligence is generally beneficialNo, an ASI would be smart enough to under-stand what we really want
What are pro’s and con’s of taking AI x-risk seriously?Pro: We need to take it seriously to do what is necessary and prevent worst-case scenariosPro: We need to take it seriously to do what is necessary and prevent worst-case scenariosContra: Being too cautious stifles innovation and will prevent us from reaping the benefits of AIContra: It takes away attention from the real (short-term) risks of AI
Specific talking pointsWe need to be humble, cannot simply assume that ASI will be safe or impossibleI have been working in AI for a long time and was convinced that ASI is a long way off, but I changed my mind after ChatGPT/​ GPT-4Yes, current technology could go wrong, but we can and will prevent that. AI development should be open-source.This is all just hype/​science fiction, there is no evidence for ASI/​x-risks, people have always been afraid of technology

My heavily biased summary of the discussion: While Bengio and Tegmark argue based on two decades of alignment research, LeCun and Mitchell merely offer heuristics of the kind “people were scared about technology in the past and all went well, so there is no need to be scared now”, an almost ridiculous optimism on the side of LeCun (along the lines of “We will not be stupid enough to build dangerous AI”, “We will be able to build benevolent ASI by iteratively improving it”) and an arrogant dismissiveness by Mitchell towards people like Yoshua Bengio and Geoffrey Hinton, calling their concerns “ungrounded speculations” and even “dangerous”. Neither Mitchell nor LeCun seem very familiar with standard AI safety topics, like instrumental goals and the orthogonality thesis, let alone agentic theory.

Not much surprise here. But one thing becomes apparent: Bengio and Tegmark are of almost identical mindsets, while LeCun and Mitchell have different opinions on many topics. Somewhat paradoxically, this may have helped the LeCun/​Mitchell side in various ways (the following is highly speculative):

  • Mitchell’s dismissiveness may have shifted the Overton window of the audience towards the possibility that Bengio and Tegmark, despite their credits, might be somehow deluded.

  • This may have strengthened LeCun’s more reasonable (in my view) stance: he admitted that ASI is possible and could pose a risk, but at the same time dismissed an x-risk on the grounds that no one would be stupid enough to build an unsafe ASI.

  • LeCun came across as somewhat “in the middle of the spectrum” between Mitchell’s total x-risk dismissal and Tegmark’s and Bengio’s pro x-risk stance, so people unsure about the issue may have taken his side.

  • Mitchell attacked Bengio and Tegmark indirectly multiple times, calling their opinions unscientific, ungrounded speculation, science fiction, etc. In contrast, Bengio and Tegmark were always respectful and polite, even when Tegmark challenged Mitchell. This may have further increased LeCun’s credibility, since there were no attacks on him and he didn’t attack anyone himself.

Although Bengio and Tegmark did a good job at explaining AI safety in layman’s terms, their arguments were probably a bit difficult to grasp for people with no prior knowledge of AI safety. Mitchell’s counter-heuristics, on the other hand (“people have always been afraid of technology”, “don’t trust the media when they hype a problem”), are familiar to almost anyone. Therefore, the debate may have appeared balanced to outsiders, when at least to me it is obvious that one side was arguing grounded in science and rationality, while the other was not.

I have drawn a few lessons for my own work and would be interested in your comments on these:

  • Explaining AI safety to the general public is even more important than I previously thought, if only to strengthen the arguments of the leading AI safety researchers in similar situations.

  • We cannot rely on logical arguments alone. We need to actively address the counter-heuristics and make it clear why they are not applicable and misleading.

  • It may be a good idea to enter such a debate with a specific framework to build your arguments on. For example, Tegmark or Bengio could have mentioned orthogonality and instrumental goals right from the start and refer to that framework whenever LeCun and Mitchell were arguing that ASI would have no reason to do bad things, or intelligence was always beneficial. I personally would probably have used a frame I call the “game of dominance”, which I use to explain why AI doesn’t have to be human-like or ASI to become uncontrollable.

  • It seems like a good idea to have a mix of differing opinions on your side, even somewhat extreme (though grounded in rationality) positions – these will strengthen the more moderate stances. In this specific case, a combination of Bengio and e.g. Yudkowsky may have been more effective.

  • Being polite and respectful is important. While Mitchell’s dismissiveness may have helped LeCun, it probably hurt her own reputation, both in the AI safety community and in the general public.

As a final remark, I would like to mention that my personal impression of Yann LeCun did improve while watching the debate. I don’t think he is right in his optimistic views (and I’m not even sure if this optimism is his true belief, or just due to his job as chief AI scientist at Meta), but at least he recognizes the enormous power of advanced AI and admits that there are certain things that must not be done.

Crossposted from LessWrong (106 points, 53 comments)