It’s possible much of that supposed additional complexity isn’t useful
Yup! That’s where I’d put my money.
It’s a forgone conclusion that a real-world system has tons of complexity that is not related to the useful functions that the system performs. Consider, for example, the silicon transistors that comprise digital chips—”the useful function that they perform” is a little story involving words like “ON” and “OFF”, but “the real-world transistor” needs three equations involving 22 parameters, to a first approximation!
By the same token, my favorite paper on the algorithmic role of dendritic computation has them basically implementing a simple set of ANDs and ORs on incoming signals. It’s quite likely that dendrites do other things too besides what’s in that one paper, but I think that example is suggestive.
Caveat: I’m mainly thinking of the complexity of understanding the neuronal algorithms involved in “human intelligence” (e.g. common sense, science, language, etc.), which (I claim) are mainly in the cortex and thalamus. I think those algorithms need to be built out of really specific and legible operations, and such operations are unlikely to line up with the full complexity of the input-output behavior of neurons. I think the claim “the useful function that a neuron performs is simpler than the neuron itself” is always true, but it’s very strongly true for “human intelligence” related algorithms, whereas it’s less true in other contexts, including probably some brainstem circuits, and the neurons in microscopic worms. It seems to me that microscopic worms just don’t have enough neurons to not squeeze out useful functionality from every squiggle in their neurons’ input-output relations. And moreover here we’re not talking about massive intricate beautifully-orchestrated learning algorithms, but rather things like “do this behavior a bit less often when the temperature is low” etc. See my post Building brain-inspired AGI is infinitely easier than understanding the brain for more discussion kinda related to this.
Addendum: In the other direction, one could point out that the authors were searching for “an approximation of an approximation of a neuron”, not “an approximation of a neuron”. (insight stolen from here.) Their ground truth was a fancier neuron model, not a real neuron. Even the fancier model is a simplification of real life. For example, if I recall correctly, neurons have been observed to do funny things like store state variables via changes in gene expression. Even the fancier model wouldn’t capture that. As in my parent comment, I think these kinds of things are highly relevant to simulating worms, and not terribly relevant to reverse-engineering the algorithms underlying human intelligence.
Yup! That’s where I’d put my money.
It’s a forgone conclusion that a real-world system has tons of complexity that is not related to the useful functions that the system performs. Consider, for example, the silicon transistors that comprise digital chips—”the useful function that they perform” is a little story involving words like “ON” and “OFF”, but “the real-world transistor” needs three equations involving 22 parameters, to a first approximation!
By the same token, my favorite paper on the algorithmic role of dendritic computation has them basically implementing a simple set of ANDs and ORs on incoming signals. It’s quite likely that dendrites do other things too besides what’s in that one paper, but I think that example is suggestive.
Caveat: I’m mainly thinking of the complexity of understanding the neuronal algorithms involved in “human intelligence” (e.g. common sense, science, language, etc.), which (I claim) are mainly in the cortex and thalamus. I think those algorithms need to be built out of really specific and legible operations, and such operations are unlikely to line up with the full complexity of the input-output behavior of neurons. I think the claim “the useful function that a neuron performs is simpler than the neuron itself” is always true, but it’s very strongly true for “human intelligence” related algorithms, whereas it’s less true in other contexts, including probably some brainstem circuits, and the neurons in microscopic worms. It seems to me that microscopic worms just don’t have enough neurons to not squeeze out useful functionality from every squiggle in their neurons’ input-output relations. And moreover here we’re not talking about massive intricate beautifully-orchestrated learning algorithms, but rather things like “do this behavior a bit less often when the temperature is low” etc. See my post Building brain-inspired AGI is infinitely easier than understanding the brain for more discussion kinda related to this.
Addendum: In the other direction, one could point out that the authors were searching for “an approximation of an approximation of a neuron”, not “an approximation of a neuron”. (insight stolen from here.) Their ground truth was a fancier neuron model, not a real neuron. Even the fancier model is a simplification of real life. For example, if I recall correctly, neurons have been observed to do funny things like store state variables via changes in gene expression. Even the fancier model wouldn’t capture that. As in my parent comment, I think these kinds of things are highly relevant to simulating worms, and not terribly relevant to reverse-engineering the algorithms underlying human intelligence.