I don’t find the Turing test evidence as convincing as you present it here.
Fair enough, I did not actually read the paper! I have talked to LLMs about consciousness and to me they seem pretty good at talking about it.
I agree that if each token you read is generated by a single forward pass through a network of fixed weights, then it seems hard to imagine how there could be any ‘inner life’ behind the words. There is no introspection. But this is not how the new generation of reasoning models work. They create a ‘chain of thought’ before producing an answer, which looks a lot like introspection if you read it!
The chain of thought is still generated via feed-forward next token prediction, right?
A commenter on my blog suggested that LLMs could still be doing enough internally that they are conscious even while generating only one token at a time, which sounds reasonable to me.
The chain of thought is still generated via feed-forward next token prediction, right?
Yes, it is.. But it still feels different to me.
If it’s possible to create consciousness on a computer at all, then at some level it will have to consist of mechanical operations which can’t by themselves be conscious. This is because you could ultimately understand what it is doing as a set of simple instructions being carried out on a processor. So although I can’t see how a single forward pass through a neural network could involve consciousness, I don’t think a larger system being built out of these operations should rule out that larger system being conscious.
In a non-reasoning model, each token in the output is generated spontaneously, which means I can’t see how there could be any conscious deliberation behind it. For example, it can’t decide to spend longer thinking about a hard problem than an easier one, in the way a human might. I find it hard to get my head around a conscioussness that can’t do that.
In a reasoning model, none of this applies.
(Although it’s true that the distinction probably isn’t quite as clear cut as I’m making out. A non-reasoning model could still decide to use its output to write out “chain of thought” style reasoning, for example.)
Yes, it could well be that an LLM isn’t conscious on a single pass, but it becomes conscious across multiple passes.
This is analogous to the Chinese room argument, but I don’t take the Chinese room argument as a reductio ad absurdum—unless you’re a substance dualist or a panpsychist, I think you have to believe that a conscious being is made up of parts that are not themselves conscious.
(And even under panpsychism I think you still have to believe that the composed being is conscious in a way that the individual parts aren’t? Not sure.)
Fair enough, I did not actually read the paper! I have talked to LLMs about consciousness and to me they seem pretty good at talking about it.
The chain of thought is still generated via feed-forward next token prediction, right?
A commenter on my blog suggested that LLMs could still be doing enough internally that they are conscious even while generating only one token at a time, which sounds reasonable to me.
Yes, it is.. But it still feels different to me.
If it’s possible to create consciousness on a computer at all, then at some level it will have to consist of mechanical operations which can’t by themselves be conscious. This is because you could ultimately understand what it is doing as a set of simple instructions being carried out on a processor. So although I can’t see how a single forward pass through a neural network could involve consciousness, I don’t think a larger system being built out of these operations should rule out that larger system being conscious.
In a non-reasoning model, each token in the output is generated spontaneously, which means I can’t see how there could be any conscious deliberation behind it. For example, it can’t decide to spend longer thinking about a hard problem than an easier one, in the way a human might. I find it hard to get my head around a conscioussness that can’t do that.
In a reasoning model, none of this applies.
(Although it’s true that the distinction probably isn’t quite as clear cut as I’m making out. A non-reasoning model could still decide to use its output to write out “chain of thought” style reasoning, for example.)
Yes, it could well be that an LLM isn’t conscious on a single pass, but it becomes conscious across multiple passes.
This is analogous to the Chinese room argument, but I don’t take the Chinese room argument as a reductio ad absurdum—unless you’re a substance dualist or a panpsychist, I think you have to believe that a conscious being is made up of parts that are not themselves conscious.
(And even under panpsychism I think you still have to believe that the composed being is conscious in a way that the individual parts aren’t? Not sure.)