I notice that I’m confused.”That is, the fact that they can reproduce Python strings shows that they are not informative about the grammar of Python (or natural language).”I don’t see how the first item here shows the second item. Reproducing Python strings clearly isn’t a sufficient condition on its own for “The model understands the grammar of Python”, but how would it be an argument against the model understanding Python? Surely a model that can reproduce Python strings is at least more likely to understand the grammar of Python than a model that cannot?The contraposition of that statement would be “If a language model is informative about the grammar of Python, it cannot reproduce Python strings.” This seems pretty clearly false to me, unless I’m missing something.
Sorry my sentence was not sufficiently well specified. What about:
“The fact that a language model can reproduce Python strings without any operations that are isomorphic to the production rules in the BNF specification of Python grammar shows that they are not informative about Python grammar.”
Now the contraposition is (I think)
“If a language model is informative about the grammar of Python then it cannot produce Python strings without any operations that are isomorphic to the production rules in the BNF.”
This then becomes the argument about implementational connectionism I made in my longer paper. If the network is implementing the BNF then it becomes far less interesting as a radical new model of Python in this case, but generally as language or cognition.