Final final edit: Congrats on the ARC-AGI-PUB results, really impressive :)
This will be my final response on this thread, because life is very time consuming and Iām rapidly reaching the point where I need to dive back into the technical literature and stress-test my beliefs and intuitions again. I hope Ryan and any readers have found this exchange useful/āenlightening for seeing two different perspectives hopefully have productive disagreement?
If you found my presentation of the scaling-skeptical position highly unconvincing, Iād recommend following the work and thoughts of Tan Zhi Xuan (find her on X here). One of biggest updates was finding her work after she pushed back on Jacob Steinhardt here, and recently she gave a talk about her approach to Alignment. I urge readers to consider spending much more of their time listening to her than to me about AI.
I feel like this is a pretty strange way to draw the line about what counts as an āLLM solutionā.
I donāt think so? Again, I wouldnāt call CICERO an āLLM solutionā. Surely thereāll be some amount of scaffolding which tips over into the scaffolding being the main thing and the LLM just being a component part? Itās probably all blurry lines for sure, but I think itās important to separate āLLM only systemsā from āsystems that include LLMsā, because itās very easy to conceptual scale up the former but harder to do the latter.
Human skeptic: That wasnāt humans sending someone to the moon that was Humans + Culture + Organizations + Science sending someone to the moon! You see, humans donāt exhibit real intelligence!
I mean, you use this as a reductio, but thatās basically the theory of Distributed Cognition, and also linked to the ideas of ācollective intelligenceā, though thatās definitely not an area Iām an expert in by any means. Also reminds me a lot Chalmers and Clarksā thesis of the Extended Mind.[1]
Of course, I think actual LLM skeptics often donāt answer āNoā to the last question. They often do have something that they think is unlikely to occur with a relatively straightforward scaffold on top of an LLM (a model descended from the current LLM paradigm, perhaps trained with semi-supervised learning and RLHF).
So I canāt speak for Chollet and other LLM skeptics, and I think again LLMs+extra (or extras+LLMs) are a different beast from LLMs on their own and possibly an important crux. Here are some things I donāt think will happen in the near-ish future (on the current paradigm):
I believe an adversarial Imitation Game, where the interrogator is aware of both the AI systemās LLM-based nature and its failure modes, is unlikely to be consistently beaten in the near future.[2]
Primarily-LLM models, in my view, are highly unlikely to exhibit autopoietic behaviour or develop agentic designs independently (i.e. without prompting/ādirection by a human controller).
I donāt anticipate these models exponential increase the rate of scientific research or AI development.[3] Theyāll more likely serve as tools used by scientists and researchers themselves to frame problems, but new and novel problems will still remain difficult and be bottlenecked by the real world + Hofstadterās law.
I donāt anticipate Primarily-LLM models to become good at controlling and manoeuvring robotic bodies in the 3D world. This is especially true in a novel-test-case scenario (if someone could make a physical equivalent of ARC to test this, thatād be great)
This would be even less likely if the scaffolding remained minimal. For instance, if thereās no initial sorting code explicitly stating [IF challenge == turing_test GO TO turing_test_game_module].
Finally, as an anti-RSI operationalisation, the idea of LLM-based models assisting in designing and constructing a Dyson Sphere within 15 years seemsā¦ particularly far-fetched for me.
Iām not sure if this reply was my best, it felt a little all-over-the-place, but we are touching on some deep or complex topics! So Iāll respectfully bow out now, and thank again for the disucssion and giving me so much to think about. I really appreciate it Ryan :)
Of course, with a new breakthrough, all bets could be off, but itās also definitionally impossible to predict those, and unrobust to draw straight lines and graphs to predict the future if you think breakthroughs will be need. (Not saying you do this, but some other AIXR people definitely seem to be)
Final final edit: Congrats on the ARC-AGI-PUB results, really impressive :)
This will be my final response on this thread, because life is very time consuming and Iām rapidly reaching the point where I need to dive back into the technical literature and stress-test my beliefs and intuitions again. I hope Ryan and any readers have found this exchange useful/āenlightening for seeing two different perspectives hopefully have productive disagreement?
If you found my presentation of the scaling-skeptical position highly unconvincing, Iād recommend following the work and thoughts of Tan Zhi Xuan (find her on X here). One of biggest updates was finding her work after she pushed back on Jacob Steinhardt here, and recently she gave a talk about her approach to Alignment. I urge readers to consider spending much more of their time listening to her than to me about AI.
I donāt think so? Again, I wouldnāt call CICERO an āLLM solutionā. Surely thereāll be some amount of scaffolding which tips over into the scaffolding being the main thing and the LLM just being a component part? Itās probably all blurry lines for sure, but I think itās important to separate āLLM only systemsā from āsystems that include LLMsā, because itās very easy to conceptual scale up the former but harder to do the latter.
I mean, you use this as a reductio, but thatās basically the theory of Distributed Cognition, and also linked to the ideas of ācollective intelligenceā, though thatās definitely not an area Iām an expert in by any means. Also reminds me a lot Chalmers and Clarksā thesis of the Extended Mind.[1]
So I canāt speak for Chollet and other LLM skeptics, and I think again LLMs+extra (or extras+LLMs) are a different beast from LLMs on their own and possibly an important crux. Here are some things I donāt think will happen in the near-ish future (on the current paradigm):
I believe an adversarial Imitation Game, where the interrogator is aware of both the AI systemās LLM-based nature and its failure modes, is unlikely to be consistently beaten in the near future.[2]
Primarily-LLM models, in my view, are highly unlikely to exhibit autopoietic behaviour or develop agentic designs independently (i.e. without prompting/ādirection by a human controller).
I donāt anticipate these models exponential increase the rate of scientific research or AI development.[3] Theyāll more likely serve as tools used by scientists and researchers themselves to frame problems, but new and novel problems will still remain difficult and be bottlenecked by the real world + Hofstadterās law.
I donāt anticipate Primarily-LLM models to become good at controlling and manoeuvring robotic bodies in the 3D world. This is especially true in a novel-test-case scenario (if someone could make a physical equivalent of ARC to test this, thatād be great)
This would be even less likely if the scaffolding remained minimal. For instance, if thereās no initial sorting code explicitly stating [IF challenge == turing_test GO TO turing_test_game_module].
Finally, as an anti-RSI operationalisation, the idea of LLM-based models assisting in designing and constructing a Dyson Sphere within 15 years seemsā¦ particularly far-fetched for me.
Iām not sure if this reply was my best, it felt a little all-over-the-place, but we are touching on some deep or complex topics! So Iāll respectfully bow out now, and thank again for the disucssion and giving me so much to think about. I really appreciate it Ryan :)
Then you get into ideas like embodiment/āenactivism etc
I can think of a bunch of strategies to win here, but Iām not gonna say so it doesnāt end up in GPT-5 or 6ā²s training data!
Of course, with a new breakthrough, all bets could be off, but itās also definitionally impossible to predict those, and unrobust to draw straight lines and graphs to predict the future if you think breakthroughs will be need. (Not saying you do this, but some other AIXR people definitely seem to be)