I don’t think I quite follow what you consider to be the reductio. In particular, I don’t see why your argument wouldn’t also go through with humans. Why doesn’t the following hold?
Biological Learning (BL) is an alternative form of computation that does not involve classical symbol systems, but instead just a bunch of neurons and some wet stuff, and its amazing success at producing human intelligence shows that human intelligence is not based on classical symbolic systems
If you claim it’s impossible for a non-classical system to create something symbolic, I don’t think you get to hide behind “we don’t know how human cognition works”. I think you been to defend the position that human cognition must be symbolic, and then explain how this arises from biological neural networks but not artificial ones.
Yes, humans made Python because we have the ability for symbolic thought.
And I am not saying that non-classical systems can’t create something symbolic. In fact this is the crux of my argument that Symbolic-Neuro symbolic architectures (see my first post) DO create symbol strings. It is the process with which they create the strings that is in question.
If you agree that bundles of biological neurons can have the capacity for symbolic thought, and that non-classical systems can create something symbolic, I don’t understand why you think anything you’ve said shows that DL cannot scale to AGI, even granting your unstated assumption that symbolic thought is necessary for AGI.
(I think that last assumption is false, but don’t think it’s a crux here so I’m keen to grant it for now, and only discuss once we’ve cleared up the other thing)
Biological neutrons have very different properties from artificial networks in very many ways. These are well documented. I would never deny that ensembles of biological neutrons have the capacity for symbol manipulation.
I also believe that non-classical systems can learn mappings between symbols, because this is in fact what they do. Language models map from word tokens to word tokens.
What they don’t do, as the inventors of DL insist, is learn classical symbol manipulation with rules defined over symbols.
Could you mechanistically explain how any of the ‘very many ways’ biological neurons are different mean that the the capacity for symbol manipulation is unique to them?
They’re obviously very different, but what I don’t think you’ve done is show that the differences are responsible for the impossibility of symbolic manipulation in artificial neural networks.
I think I may have said something to confuse the issue. Artificial neural networks certainly ARE capable of representing classical symbolic computations. In fact the first neural networks (e.g. perceptron) did just that. They typically do that with local representations where individual nodes assume the role of representing a given variable. But these were not very good at other tasks like generalisation.
More advanced distributed networks emerged with DL being the newest incarnation. These have representations which makes it very difficult (if not impossible) to dedicate nodes to variables. Which does not worry the architects because they specifically believe that the non-localised representation is what makes them so powerful (see Bengio, LeCun and Hinton’s article for their Turing award)
Turning to real neurons, the fact is that we really don’t know all that much about how they represent knowledge. We know where they tend to fire in response to given stimuli, we know how they are connected, and we know that they have some hierarchical representations. So I can’t give you a biological explanation of how neural ensembles can represent variables. All I can do is give you arguments that humans DO perform symbolic manipulation on variables, so somehow their brain has to be able to encode this.
If you can make an artificial network somehow do this eventually then fine. I will support those efforts. But we are nowhere near that, and the main actors are not even pushing in that direction.
That last comment seems very far from the original post which claimed
We have no good reason, only faith and marketing, to believe that we will accomplish AGI by pursuing the DL based AI route.
If we don’t have a biological representation of how BNNs can represent and perform symbolic representation, why do we have reason to believe that we know ANNs can’t?
Without an ability to point to the difference, this isn’t anything close to a reductio, it’s just saying “yeah I don’t buy it dude, I don’t reckon AI will be that good”
Sorry I think you are misunderstanding the reductio argument. That argument simply undermines the claim that natural language is not based on a generative phrase structure grammar. That is, that non symbolic DL is the “proper” model of language. In fact they are called “language models”. I claim they are not models of language, and therefore there is no reason to discard symbolic models … which is where the need for symbol manipulation comes from. Hence a very different sort of architecture than current DL
And of course we can point to the difference between artificial and biological networks. I didn’t because there are too many! One of the big ones is back propagation. THE major reason we have ANNs in the first place, completely implausible biologically. No back propagation in the brain.
I can’t fully understand or engage with your claims or posts, because I don’t actually know how AI and “symbolic logic” would work, how it reasons about anything, and really even how to start thinking about it.
Can you provide a primer of what symbolic logic/symbolic computing is, as it is relevant to AI (in any sense), and how it is supposed to work on a detailed level, i.e., so I could independently apply it to problems? (E.g. blog post, PDF chapter of a book).
(Assume your audience knows statistical machine learning, like linear classifiers, deep learning, rule based systems, coding, basic math, etc.).
Charles, I don’t think it is necessary to understand all the details about logic to understand my point. The example of a truth table is enough, as I explain in my first post.
It’s more like these deep learning systems are mimicking Python very well [1]. There’s no actual symbolic reasoning. You believe this...right?
Zooming out and untangling this a bit, I think the following is a bit closer to the issue?
Deep Learning (DL) is an alternative form of computation that does not involve classical symbol systems, and its amazing success shows that human intelligence is not based on classical symbolic systems. In fact, Geoff Hinton in his Turing Award Speech proclaimed that “the success of machine translation is the last nail in the coffin of symbolic AI”.
Why is this right?
There’s no reason think that any particular computational performance is connected to human intelligence. Why do you believe this? A smartphone is amazingly better than humans at a lot of tasks but that doesn’t seem to mean anything obvious about the nature of human intelligence.
Zooming out more here, it reads like there’s some sort of beef/framework/grand theory/assertion related to symbolic logic, human intelligence, and AGI that you are strongly engaged in. It reads like you got really into this theory and built up your own argument, but it’s unclear why the claims of this underlying theory are true (or even what they are).
The resulting argument has a lot of nested claims and red herrings (the Python thing) and it’s hard to untangle.
I don’t think the question of whether intelligence is pattern recognition, or symbolic logic, is the essence of people’s concerns about AGI. Do you agree or not?
I’m not sure this statement is correct or meaningful (in the context of your argument) because learning Python syntactically isn’t what’s hard, but expressing logic in Python is, and I don’t know what this expression of logic means in your theory. I don’t think you addressed it and I can’t really fill in where it fits in your theory.
Charles, you are right, there is a deep theoretical “beef” behind the issues, but it is not my beef. The debate between “connectionist” neural network theories and symbol based theories raged very much in the 1980s, 1990s. These were really nice scientific debates based on empirical results. Connectionism faded away because it did not prove to be adequate in explaining a lot of challenges. Geoff Hinton was a big part of that debate.
When compute power and data availability grew so fantastically in the 2010s, DL started to have practical success as you see today. Hinton re emerged victoriously and has been wildly attacking believers in symbolic systems ever since. In fact there is a video of him deriding the EU for being tricked into continued funding of symbolic AI research!
I prefer to stay with scientific argumentation and claim that the fact that DL can produce Python defeats Hinton’s claim (not mine) that DL machine translation proves that language is not a symbolic process.
I literally read your post for over ~30 minutes to try to figure out what is going on. I don’t think what I wrote above is relevant/the issue anymore.
Basically, I think what you did was write a narration to yourself, with things that are individually basically true, but that no one claims is important. You also slip in claims like “human cognition must resemble AGI for AGI to happen”, but without making a tight argument[1].
You then point this resulting reasoning at your final point: “We have no good reason, only faith and marketing, to believe that we will accomplish AGI by pursuing the DL based AI route.”.
Also, it’s really hard to follow this, there’s things in this argument that seem to be like a triple negative.
Honestly, both my decision to read this and my subsequent performance in untangling this, makes me think I’m pretty dumb.
For example, you say that “DL is much more successful than symbolic AI because it’s closer to the human brain”, and you say this is “defeated” later. Ok. That seems fine.
Later you “defeat” the claim that:
“DL-symbol systems (whatever those are) will be much better because DL has already shown that classical symbol systems are not the right way to model cognitive abilities.”
You say this means:
We don’t know that DL-symbol systems (whatever those are) will be much better than classical AI because DL has not shown anything about the nature of human cognition.
But no one is talking about the nature of human cognition being related to AI?
This is your final point before claiming that AGI can’t come from DL or “symbol-DL”.
Charles, thanks for spending so much time trying to understand my argument. I hope my previous answer helps. Also I added a paragraph to clarify my stance before I give my points.
Also you say that “You also slip in claims like “human cognition must resemble AGI for AGI to happen”″. I don’t think I said that. If I did I must correct it.
I don’t think I quite follow what you consider to be the reductio. In particular, I don’t see why your argument wouldn’t also go through with humans. Why doesn’t the following hold?
Biological Learning (BL) is an alternative form of computation that does not involve classical symbol systems, but instead just a bunch of neurons and some wet stuff, and its amazing success at producing human intelligence shows that human intelligence is not based on classical symbolic systems
The reductio is specifically about Python. I show that the argument must conclude that Python is not symbolic, which means the argument must be wrong.
So your alterenative would be that BL shows that Python is not based on classical symbol systems.
We don’t know how human cognition works which is why the BL argument is appealing. But we do know how Python works.
But humans made python.
If you claim it’s impossible for a non-classical system to create something symbolic, I don’t think you get to hide behind “we don’t know how human cognition works”. I think you been to defend the position that human cognition must be symbolic, and then explain how this arises from biological neural networks but not artificial ones.
Yes, humans made Python because we have the ability for symbolic thought.
And I am not saying that non-classical systems can’t create something symbolic. In fact this is the crux of my argument that Symbolic-Neuro symbolic architectures (see my first post) DO create symbol strings. It is the process with which they create the strings that is in question.
If you agree that bundles of biological neurons can have the capacity for symbolic thought, and that non-classical systems can create something symbolic, I don’t understand why you think anything you’ve said shows that DL cannot scale to AGI, even granting your unstated assumption that symbolic thought is necessary for AGI.
(I think that last assumption is false, but don’t think it’s a crux here so I’m keen to grant it for now, and only discuss once we’ve cleared up the other thing)
Biological neutrons have very different properties from artificial networks in very many ways. These are well documented. I would never deny that ensembles of biological neutrons have the capacity for symbol manipulation.
I also believe that non-classical systems can learn mappings between symbols, because this is in fact what they do. Language models map from word tokens to word tokens.
What they don’t do, as the inventors of DL insist, is learn classical symbol manipulation with rules defined over symbols.
Could you mechanistically explain how any of the ‘very many ways’ biological neurons are different mean that the the capacity for symbol manipulation is unique to them?
They’re obviously very different, but what I don’t think you’ve done is show that the differences are responsible for the impossibility of symbolic manipulation in artificial neural networks.
I think I may have said something to confuse the issue. Artificial neural networks certainly ARE capable of representing classical symbolic computations. In fact the first neural networks (e.g. perceptron) did just that. They typically do that with local representations where individual nodes assume the role of representing a given variable. But these were not very good at other tasks like generalisation.
More advanced distributed networks emerged with DL being the newest incarnation. These have representations which makes it very difficult (if not impossible) to dedicate nodes to variables. Which does not worry the architects because they specifically believe that the non-localised representation is what makes them so powerful (see Bengio, LeCun and Hinton’s article for their Turing award)
Turning to real neurons, the fact is that we really don’t know all that much about how they represent knowledge. We know where they tend to fire in response to given stimuli, we know how they are connected, and we know that they have some hierarchical representations. So I can’t give you a biological explanation of how neural ensembles can represent variables. All I can do is give you arguments that humans DO perform symbolic manipulation on variables, so somehow their brain has to be able to encode this.
If you can make an artificial network somehow do this eventually then fine. I will support those efforts. But we are nowhere near that, and the main actors are not even pushing in that direction.
That last comment seems very far from the original post which claimed
If we don’t have a biological representation of how BNNs can represent and perform symbolic representation, why do we have reason to believe that we know ANNs can’t?
Without an ability to point to the difference, this isn’t anything close to a reductio, it’s just saying “yeah I don’t buy it dude, I don’t reckon AI will be that good”
Sorry I think you are misunderstanding the reductio argument. That argument simply undermines the claim that natural language is not based on a generative phrase structure grammar. That is, that non symbolic DL is the “proper” model of language. In fact they are called “language models”. I claim they are not models of language, and therefore there is no reason to discard symbolic models … which is where the need for symbol manipulation comes from. Hence a very different sort of architecture than current DL
And of course we can point to the difference between artificial and biological networks. I didn’t because there are too many! One of the big ones is back propagation. THE major reason we have ANNs in the first place, completely implausible biologically. No back propagation in the brain.
You seem really informed about detailed aspects of language, modelling, and seem to be an active researcher with a long career in modelling and reasoning.
I can’t fully understand or engage with your claims or posts, because I don’t actually know how AI and “symbolic logic” would work, how it reasons about anything, and really even how to start thinking about it.
Can you provide a primer of what symbolic logic/symbolic computing is, as it is relevant to AI (in any sense), and how it is supposed to work on a detailed level, i.e., so I could independently apply it to problems? (E.g. blog post, PDF chapter of a book).
(Assume your audience knows statistical machine learning, like linear classifiers, deep learning, rule based systems, coding, basic math, etc.).
Charles, I don’t think it is necessary to understand all the details about logic to understand my point. The example of a truth table is enough, as I explain in my first post.
It’s more like these deep learning systems are mimicking Python very well [1]. There’s no actual symbolic reasoning. You believe this...right?
Zooming out and untangling this a bit, I think the following is a bit closer to the issue?
Why is this right?
There’s no reason think that any particular computational performance is connected to human intelligence. Why do you believe this? A smartphone is amazingly better than humans at a lot of tasks but that doesn’t seem to mean anything obvious about the nature of human intelligence.
Zooming out more here, it reads like there’s some sort of beef/framework/grand theory/assertion related to symbolic logic, human intelligence, and AGI that you are strongly engaged in. It reads like you got really into this theory and built up your own argument, but it’s unclear why the claims of this underlying theory are true (or even what they are).
The resulting argument has a lot of nested claims and red herrings (the Python thing) and it’s hard to untangle.
I don’t think the question of whether intelligence is pattern recognition, or symbolic logic, is the essence of people’s concerns about AGI. Do you agree or not?
I’m not sure this statement is correct or meaningful (in the context of your argument) because learning Python syntactically isn’t what’s hard, but expressing logic in Python is, and I don’t know what this expression of logic means in your theory. I don’t think you addressed it and I can’t really fill in where it fits in your theory.
Charles, you are right, there is a deep theoretical “beef” behind the issues, but it is not my beef. The debate between “connectionist” neural network theories and symbol based theories raged very much in the 1980s, 1990s. These were really nice scientific debates based on empirical results. Connectionism faded away because it did not prove to be adequate in explaining a lot of challenges. Geoff Hinton was a big part of that debate.
When compute power and data availability grew so fantastically in the 2010s, DL started to have practical success as you see today. Hinton re emerged victoriously and has been wildly attacking believers in symbolic systems ever since. In fact there is a video of him deriding the EU for being tricked into continued funding of symbolic AI research!
I prefer to stay with scientific argumentation and claim that the fact that DL can produce Python defeats Hinton’s claim (not mine) that DL machine translation proves that language is not a symbolic process.
I literally read your post for over ~30 minutes to try to figure out what is going on. I don’t think what I wrote above is relevant/the issue anymore.
Basically, I think what you did was write a narration to yourself, with things that are individually basically true, but that no one claims is important. You also slip in claims like “human cognition must resemble AGI for AGI to happen”, but without making a tight argument[1].
You then point this resulting reasoning at your final point: “We have no good reason, only faith and marketing, to believe that we will accomplish AGI by pursuing the DL based AI route.”.
Also, it’s really hard to follow this, there’s things in this argument that seem to be like a triple negative.
Honestly, both my decision to read this and my subsequent performance in untangling this, makes me think I’m pretty dumb.
For example, you say that “DL is much more successful than symbolic AI because it’s closer to the human brain”, and you say this is “defeated” later. Ok. That seems fine.
Later you “defeat” the claim that:
You say this means:
But no one is talking about the nature of human cognition being related to AI?
This is your final point before claiming that AGI can’t come from DL or “symbol-DL”.
Charles, thanks for spending so much time trying to understand my argument. I hope my previous answer helps. Also I added a paragraph to clarify my stance before I give my points.
Also you say that “You also slip in claims like “human cognition must resemble AGI for AGI to happen”″. I don’t think I said that. If I did I must correct it.