I was pleasantly surprised by this paper (given how much dross has been written on this topic). But I still have some fairly strong disagreements with it. My thoughts on the four fallacies Mitchell identifies:
Fallacy 1: Narrow intelligence is on a continuum with general intelligence
This is hard to evaluate, since Mitchell only discusses it very briefly. I do think that people underestimate the gap between solving tasks with near-infinite data (like Starcraft) vs low-data tasks. But saying that GPT-3 isn’t a step towards general intelligence also seems misguided, given the importance of few-shot learning.
Fallacy 2: Easy things are easy and hard things are hard
I agree that Moravec’s paradox is important and underrated. But this also cuts the other way: if chess and Go were easy, then we should be open to the possibility that maths and physics are too.
Fallacy 3: The lure of wishful mnemonics
This is true and important. My favourite example is artificial planning. Tree search algorithms are radically different from human planning, which operates over abstractions. Yet this is hard to see because we use the same word for both.
Fallacy 4: Intelligence is all in the brain
This is the one I disagree with most, because “embodied cognition” is a very slippery concept. What does it mean? “The representation of conceptual knowledge is … multimodal”—okay, but CLIP is multimodal.
“Thoughts are inextricably associated with perception, action, and emotion.” Okay, but RL agents have perceptions and actions. And even if the body plays a crucial role in human emotions, it’s a big leap to claim that disembodied agents therefore can’t develop emotions.
Under this fallacy, Mitchell also discusses AI safety arguments by Bostrom and Russell. I agree that early characterisations of AIs as “purely rational” were misguided. Mitchell argues that AIs will likely also have emotions, cultural biases, a strong sense of selfhood and autonomy, and a commonsense understanding of the world. This seems plausible! But note that none of these directly solves the problem of misaligned goals. Sociopaths have all these traits, but we wouldn’t want them to have superhuman intelligence.
This does raise the question: can early arguments for AI risk be reformulated to rely less on this “purely rational” characterisation? I think so—in fact, that’s what I tried to do in this report.
On Morvec’s paradox, chess and Go are quite different from math and physics. Chess and Go are both finite games (a finite number of pieces, on a finite board, with a finite number of open moves at every point in a games) with a small geometrically simple footprint. That is not true for either math or physics. Both are radically open-ended and unbounded, though math has no physical footprint at al
I was pleasantly surprised by this paper (given how much dross has been written on this topic). But I still have some fairly strong disagreements with it. My thoughts on the four fallacies Mitchell identifies:
Fallacy 1: Narrow intelligence is on a continuum with general intelligence
This is hard to evaluate, since Mitchell only discusses it very briefly. I do think that people underestimate the gap between solving tasks with near-infinite data (like Starcraft) vs low-data tasks. But saying that GPT-3 isn’t a step towards general intelligence also seems misguided, given the importance of few-shot learning.
Fallacy 2: Easy things are easy and hard things are hard
I agree that Moravec’s paradox is important and underrated. But this also cuts the other way: if chess and Go were easy, then we should be open to the possibility that maths and physics are too.
Fallacy 3: The lure of wishful mnemonics
This is true and important. My favourite example is artificial planning. Tree search algorithms are radically different from human planning, which operates over abstractions. Yet this is hard to see because we use the same word for both.
Fallacy 4: Intelligence is all in the brain
This is the one I disagree with most, because “embodied cognition” is a very slippery concept. What does it mean? “The representation of conceptual knowledge is … multimodal”—okay, but CLIP is multimodal.
“Thoughts are inextricably associated with perception, action, and emotion.” Okay, but RL agents have perceptions and actions. And even if the body plays a crucial role in human emotions, it’s a big leap to claim that disembodied agents therefore can’t develop emotions.
Under this fallacy, Mitchell also discusses AI safety arguments by Bostrom and Russell. I agree that early characterisations of AIs as “purely rational” were misguided. Mitchell argues that AIs will likely also have emotions, cultural biases, a strong sense of selfhood and autonomy, and a commonsense understanding of the world. This seems plausible! But note that none of these directly solves the problem of misaligned goals. Sociopaths have all these traits, but we wouldn’t want them to have superhuman intelligence.
This does raise the question: can early arguments for AI risk be reformulated to rely less on this “purely rational” characterisation? I think so—in fact, that’s what I tried to do in this report.
On Morvec’s paradox, chess and Go are quite different from math and physics. Chess and Go are both finite games (a finite number of pieces, on a finite board, with a finite number of open moves at every point in a games) with a small geometrically simple footprint. That is not true for either math or physics. Both are radically open-ended and unbounded, though math has no physical footprint at al