Sure, François Chollet recently changed his prediction from AGI maybe in about 10 years to AGI maybe in about 5 years, but his very incisive arguments about the shortcomings of LLMs are logically and intellectually independent from his, in my opinion, extremely dubious prediction of AGI in about 5 years. I would like people who believe LLMs will scale to AGI to seriously engage with his arguments about why it wonât. His prediction about when AGI will happen is kind of beside the point.
Thatâs basically all that needs to be said about this, but Iâll elaborate anyway because I think the details are interesting and intellectually helpful.
If I understand François Cholletâs point of view correctly, his prediction of AGI in about 5 years depends on a) program synthesis being the key to solving AGI and b) everything required for program synthesis to take deep learning the rest of the way to AGI being solved within about 5 years. I have extreme doubts about both (a) and (b) and, for that matter, I would guess most people who think AGI will come within 7 years have strong doubts about at least (a). Thinking that LLMs will scale to AGI and believing that solving program synthesis is required to achieve AGI are incompatible views.
François Chollet, Yann LeCun of FAIR, Jeff Hawkins of Numenta, and Richard Sutton of Amii and Keen Technologies are four AI researchers who have strongly criticized LLMs[1] and have proposed four different directions for AI research to get to AGI:
Reinforcement learning and the Alberta Plan for Richard Sutton
All four have also at times made statements indicating they think their preferred research roadmap to AGI can be executed on a relatively short timescale, although with some important caveats from the three other than Chollet. (Chollet might have given a caveat somewhere that I missed. I think heâs said the least on this topic overall or Iâve missed what heâs said.) I already described Cholletâs thoughts on this. As for the others:
Yann LeCun gave a nuanced and self-aware answer, saying, âFive to ten years would be if everything goes great. All the plans that weâve been making will succeed. Weâre not going to encounter unexpected obstacles, but that is almost certainly not going to happen.â In another interview, he said, âall of this is going to take at least a decade and probably much more because there are a lot of problems that weâre not seeing right now that we have not encounteredâ.
When discussing the prospect of Numenta creating a machine that can think like a human (not explicitly using the term âAGIâ), Jeff Hawkins said, âItâs not going to take decades, itâs a matter of a few years, optimistically, but I think thatâs possible.â
Richard Sutton has said he thinks thereâs a 25% chance by 2030 weâll âunderstand intelligenceâ, by which I think he means weâll attain the requisite knowledge to build AGI, but Iâm not sure. I also think when he says âweâ he specifically means himself and his colleagues, but Iâm also not sure.
So, Chollet, LeCun, Hawkins, and Sutton all think LLMs are insufficient to get to AGI. They all argue that AGI requires fundamentally different ideas than what the mainstream of AI research is focusing on â and they advocate for four completely different ideas.[2] And all four of them are optimistic that their preferred research roadmap has a realistic chance of getting to AGI within a relatively short time horizon. Isnât that interesting?
Either three out of four of them have to be wrong about the key insights needed to unlock AGI or all four them have to be wrong. (I guess the solution to AGI could also turn out to be some combination of their ideas, in which case they would be partially right and partially wrong in some combination.) Itâs interesting that all four independently, simultaneously think their respective roadmaps are viable on a roughly similar timescale when a) it would be hard to imagine a strong theoretical or intellectual reason to support the idea that we will figure out the solution to AGI soon, regardless of what the solution actually is (e.g. whether it requires simulating cortical columns or using non-deep learning AI methods or using novel deep learning methods) and b) there are psychological reasons to believe such things, like the appeal of having an ambitious but achievable goal on a timescale thatâs motivating.
I find what Yann LeCun has said about this to be the wisest and most self-aware. I canât find the interview where he said this, but he said something along the lines of (paraphrasing based on memory): over the last several decades, many people have come along and said they have the solution to AGI and theyâve all been wrong. So, if someone comes along and tells you they have the solution to AGI, you should not believe them. Iâm another person whoâs coming along and telling you I have the solution to AGI, and you should not believe me. But I still think Iâm right.
I think (or at least hope), once described like this, in the context I just gave above, that kind of skepticism will strike most people as logical and prudent.
The implication of this is that if you accept François Cholletâs arguments about why LLMs wonât scale to AGI, which I think are quite good, you should still be skeptical of his view that heâs found the solution to AGI, that itâs program synthesis, and that the research roadmap for program synthesis (leading from its current state all the way to AGI) can be completed maybe in about 5 years. By default, you should be skeptical of any claims of this kind. But that claim has no bearing on whether his arguments about the fundamental weaknesses of LLMs are correct.
Chollet believes that the major AI labs are in fact working on program synthesis, but as far as I know, this hasnât been confirmed by any lab and, if it is happening, it hasnât made its way into published research yet.
François Chollet has since adjusted his AGI timelines to 5 years.
Sure, François Chollet recently changed his prediction from AGI maybe in about 10 years to AGI maybe in about 5 years, but his very incisive arguments about the shortcomings of LLMs are logically and intellectually independent from his, in my opinion, extremely dubious prediction of AGI in about 5 years. I would like people who believe LLMs will scale to AGI to seriously engage with his arguments about why it wonât. His prediction about when AGI will happen is kind of beside the point.
Thatâs basically all that needs to be said about this, but Iâll elaborate anyway because I think the details are interesting and intellectually helpful.
If I understand François Cholletâs point of view correctly, his prediction of AGI in about 5 years depends on a) program synthesis being the key to solving AGI and b) everything required for program synthesis to take deep learning the rest of the way to AGI being solved within about 5 years. I have extreme doubts about both (a) and (b) and, for that matter, I would guess most people who think AGI will come within 7 years have strong doubts about at least (a). Thinking that LLMs will scale to AGI and believing that solving program synthesis is required to achieve AGI are incompatible views.
François Chollet, Yann LeCun of FAIR, Jeff Hawkins of Numenta, and Richard Sutton of Amii and Keen Technologies are four AI researchers who have strongly criticized LLMs[1] and have proposed four different directions for AI research to get to AGI:
Program synthesis synthesis for François Chollet
Energy-based models for Yann LeCun
The Thousand Brains Theory for Jeff Hawkins
Reinforcement learning and the Alberta Plan for Richard Sutton
All four have also at times made statements indicating they think their preferred research roadmap to AGI can be executed on a relatively short timescale, although with some important caveats from the three other than Chollet. (Chollet might have given a caveat somewhere that I missed. I think heâs said the least on this topic overall or Iâve missed what heâs said.) I already described Cholletâs thoughts on this. As for the others:
Yann LeCun gave a nuanced and self-aware answer, saying, âFive to ten years would be if everything goes great. All the plans that weâve been making will succeed. Weâre not going to encounter unexpected obstacles, but that is almost certainly not going to happen.â In another interview, he said, âall of this is going to take at least a decade and probably much more because there are a lot of problems that weâre not seeing right now that we have not encounteredâ.
When discussing the prospect of Numenta creating a machine that can think like a human (not explicitly using the term âAGIâ), Jeff Hawkins said, âItâs not going to take decades, itâs a matter of a few years, optimistically, but I think thatâs possible.â
Richard Sutton has said he thinks thereâs a 25% chance by 2030 weâll âunderstand intelligenceâ, by which I think he means weâll attain the requisite knowledge to build AGI, but Iâm not sure. I also think when he says âweâ he specifically means himself and his colleagues, but Iâm also not sure.
So, Chollet, LeCun, Hawkins, and Sutton all think LLMs are insufficient to get to AGI. They all argue that AGI requires fundamentally different ideas than what the mainstream of AI research is focusing on â and they advocate for four completely different ideas.[2] And all four of them are optimistic that their preferred research roadmap has a realistic chance of getting to AGI within a relatively short time horizon. Isnât that interesting?
Either three out of four of them have to be wrong about the key insights needed to unlock AGI or all four them have to be wrong. (I guess the solution to AGI could also turn out to be some combination of their ideas, in which case they would be partially right and partially wrong in some combination.) Itâs interesting that all four independently, simultaneously think their respective roadmaps are viable on a roughly similar timescale when a) it would be hard to imagine a strong theoretical or intellectual reason to support the idea that we will figure out the solution to AGI soon, regardless of what the solution actually is (e.g. whether it requires simulating cortical columns or using non-deep learning AI methods or using novel deep learning methods) and b) there are psychological reasons to believe such things, like the appeal of having an ambitious but achievable goal on a timescale thatâs motivating.
I find what Yann LeCun has said about this to be the wisest and most self-aware. I canât find the interview where he said this, but he said something along the lines of (paraphrasing based on memory): over the last several decades, many people have come along and said they have the solution to AGI and theyâve all been wrong. So, if someone comes along and tells you they have the solution to AGI, you should not believe them. Iâm another person whoâs coming along and telling you I have the solution to AGI, and you should not believe me. But I still think Iâm right.
I think (or at least hope), once described like this, in the context I just gave above, that kind of skepticism will strike most people as logical and prudent.
The implication of this is that if you accept François Cholletâs arguments about why LLMs wonât scale to AGI, which I think are quite good, you should still be skeptical of his view that heâs found the solution to AGI, that itâs program synthesis, and that the research roadmap for program synthesis (leading from its current state all the way to AGI) can be completed maybe in about 5 years. By default, you should be skeptical of any claims of this kind. But that claim has no bearing on whether his arguments about the fundamental weaknesses of LLMs are correct.
Yann LeCun and Richard Sutton won Turing Awards for their pioneering work in deep learning and reinforcement learning, respectively.
Chollet believes that the major AI labs are in fact working on program synthesis, but as far as I know, this hasnât been confirmed by any lab and, if it is happening, it hasnât made its way into published research yet.