In theory, the best way to be the best next-word-predictor is to model humans. Internally, humans model the world they live in. A sufficiently powerful human modeler would likely model the world the humans live in. Further, humans reason and so a really good next-word-predictor would be able to more accurately predict the next word by reasoning. Similarly, it is an optimization strategy to develop other cognitive abilities, logic, etc.
All of this allows you to predict the correct next word with less “neurons” because it takes fewer neurons to learn how to do logical deduction and memorize some premises than it takes to memorize all of the possible outputs that some future prompt may require.
The fact that we train on human data just means that we are training the AI to be able to reason and critically think in the same way we do. Once it has that ability, we can then “scale it up”, which is something humans really struggle with.
In theory, the best way to be the best next-word-predictor is to model humans. Internally, humans model the world they live in. A sufficiently powerful human modeler would likely model the world the humans live in. Further, humans reason and so a really good next-word-predictor would be able to more accurately predict the next word by reasoning. Similarly, it is an optimization strategy to develop other cognitive abilities, logic, etc.
All of this allows you to predict the correct next word with less “neurons” because it takes fewer neurons to learn how to do logical deduction and memorize some premises than it takes to memorize all of the possible outputs that some future prompt may require.
The fact that we train on human data just means that we are training the AI to be able to reason and critically think in the same way we do. Once it has that ability, we can then “scale it up”, which is something humans really struggle with.