Final final edit: Congrats on the ARC-AGI-PUB results, really impressive :)
This will be my final response on this thread, because life is very time consuming and Iâm rapidly reaching the point where I need to dive back into the technical literature and stress-test my beliefs and intuitions again. I hope Ryan and any readers have found this exchange useful/âenlightening for seeing two different perspectives hopefully have productive disagreement?
If you found my presentation of the scaling-skeptical position highly unconvincing, Iâd recommend following the work and thoughts of Tan Zhi Xuan (find her on X here). One of biggest updates was finding her work after she pushed back on Jacob Steinhardt here, and recently she gave a talk about her approach to Alignment. I urge readers to consider spending much more of their time listening to her than to me about AI.
I feel like this is a pretty strange way to draw the line about what counts as an âLLM solutionâ.
I donât think so? Again, I wouldnât call CICERO an âLLM solutionâ. Surely thereâll be some amount of scaffolding which tips over into the scaffolding being the main thing and the LLM just being a component part? Itâs probably all blurry lines for sure, but I think itâs important to separate âLLM only systemsâ from âsystems that include LLMsâ, because itâs very easy to conceptual scale up the former but harder to do the latter.
Human skeptic: That wasnât humans sending someone to the moon that was Humans + Culture + Organizations + Science sending someone to the moon! You see, humans donât exhibit real intelligence!
I mean, you use this as a reductio, but thatâs basically the theory of Distributed Cognition, and also linked to the ideas of âcollective intelligenceâ, though thatâs definitely not an area Iâm an expert in by any means. Also reminds me a lot Chalmers and Clarksâ thesis of the Extended Mind.[1]
Of course, I think actual LLM skeptics often donât answer âNoâ to the last question. They often do have something that they think is unlikely to occur with a relatively straightforward scaffold on top of an LLM (a model descended from the current LLM paradigm, perhaps trained with semi-supervised learning and RLHF).
So I canât speak for Chollet and other LLM skeptics, and I think again LLMs+extra (or extras+LLMs) are a different beast from LLMs on their own and possibly an important crux. Here are some things I donât think will happen in the near-ish future (on the current paradigm):
I believe an adversarial Imitation Game, where the interrogator is aware of both the AI systemâs LLM-based nature and its failure modes, is unlikely to be consistently beaten in the near future.[2]
Primarily-LLM models, in my view, are highly unlikely to exhibit autopoietic behaviour or develop agentic designs independently (i.e. without prompting/âdirection by a human controller).
I donât anticipate these models exponential increase the rate of scientific research or AI development.[3] Theyâll more likely serve as tools used by scientists and researchers themselves to frame problems, but new and novel problems will still remain difficult and be bottlenecked by the real world + Hofstadterâs law.
I donât anticipate Primarily-LLM models to become good at controlling and manoeuvring robotic bodies in the 3D world. This is especially true in a novel-test-case scenario (if someone could make a physical equivalent of ARC to test this, thatâd be great)
This would be even less likely if the scaffolding remained minimal. For instance, if thereâs no initial sorting code explicitly stating [IF challenge == turing_test GO TO turing_test_game_module].
Finally, as an anti-RSI operationalisation, the idea of LLM-based models assisting in designing and constructing a Dyson Sphere within 15 years seems⌠particularly far-fetched for me.
Iâm not sure if this reply was my best, it felt a little all-over-the-place, but we are touching on some deep or complex topics! So Iâll respectfully bow out now, and thank again for the disucssion and giving me so much to think about. I really appreciate it Ryan :)
Of course, with a new breakthrough, all bets could be off, but itâs also definitionally impossible to predict those, and unrobust to draw straight lines and graphs to predict the future if you think breakthroughs will be need. (Not saying you do this, but some other AIXR people definitely seem to be)
Final final edit: Congrats on the ARC-AGI-PUB results, really impressive :)
This will be my final response on this thread, because life is very time consuming and Iâm rapidly reaching the point where I need to dive back into the technical literature and stress-test my beliefs and intuitions again. I hope Ryan and any readers have found this exchange useful/âenlightening for seeing two different perspectives hopefully have productive disagreement?
If you found my presentation of the scaling-skeptical position highly unconvincing, Iâd recommend following the work and thoughts of Tan Zhi Xuan (find her on X here). One of biggest updates was finding her work after she pushed back on Jacob Steinhardt here, and recently she gave a talk about her approach to Alignment. I urge readers to consider spending much more of their time listening to her than to me about AI.
I donât think so? Again, I wouldnât call CICERO an âLLM solutionâ. Surely thereâll be some amount of scaffolding which tips over into the scaffolding being the main thing and the LLM just being a component part? Itâs probably all blurry lines for sure, but I think itâs important to separate âLLM only systemsâ from âsystems that include LLMsâ, because itâs very easy to conceptual scale up the former but harder to do the latter.
I mean, you use this as a reductio, but thatâs basically the theory of Distributed Cognition, and also linked to the ideas of âcollective intelligenceâ, though thatâs definitely not an area Iâm an expert in by any means. Also reminds me a lot Chalmers and Clarksâ thesis of the Extended Mind.[1]
So I canât speak for Chollet and other LLM skeptics, and I think again LLMs+extra (or extras+LLMs) are a different beast from LLMs on their own and possibly an important crux. Here are some things I donât think will happen in the near-ish future (on the current paradigm):
I believe an adversarial Imitation Game, where the interrogator is aware of both the AI systemâs LLM-based nature and its failure modes, is unlikely to be consistently beaten in the near future.[2]
Primarily-LLM models, in my view, are highly unlikely to exhibit autopoietic behaviour or develop agentic designs independently (i.e. without prompting/âdirection by a human controller).
I donât anticipate these models exponential increase the rate of scientific research or AI development.[3] Theyâll more likely serve as tools used by scientists and researchers themselves to frame problems, but new and novel problems will still remain difficult and be bottlenecked by the real world + Hofstadterâs law.
I donât anticipate Primarily-LLM models to become good at controlling and manoeuvring robotic bodies in the 3D world. This is especially true in a novel-test-case scenario (if someone could make a physical equivalent of ARC to test this, thatâd be great)
This would be even less likely if the scaffolding remained minimal. For instance, if thereâs no initial sorting code explicitly stating [IF challenge == turing_test GO TO turing_test_game_module].
Finally, as an anti-RSI operationalisation, the idea of LLM-based models assisting in designing and constructing a Dyson Sphere within 15 years seems⌠particularly far-fetched for me.
Iâm not sure if this reply was my best, it felt a little all-over-the-place, but we are touching on some deep or complex topics! So Iâll respectfully bow out now, and thank again for the disucssion and giving me so much to think about. I really appreciate it Ryan :)
Then you get into ideas like embodiment/âenactivism etc
I can think of a bunch of strategies to win here, but Iâm not gonna say so it doesnât end up in GPT-5 or 6â˛s training data!
Of course, with a new breakthrough, all bets could be off, but itâs also definitionally impossible to predict those, and unrobust to draw straight lines and graphs to predict the future if you think breakthroughs will be need. (Not saying you do this, but some other AIXR people definitely seem to be)