I agree there is a non-negligible chance that existing LLMs are already conscious, and I think this is a really interesting and important discussion to have. Thanks for writing it up! I donât think I would put the chances as high as 10% though.
I donât find the Turing test evidence as convincing as you present it here. The paper you cited released their test online for people to try. I played it quite a lot, and I was always able to distinguish the human from the AI (they donât tell you which AI you are paired with, but presumably some of those were with GPT-4.5).
On the other hand, I donât find your strongest argument against LLM consciousness to be as convincing either. I agree that if each token you read is generated by a single forward pass through a network of fixed weights, then it seems hard to imagine how there could be any âinner lifeâ behind the words. There is no introspection. But this is not how the new generation of reasoning models work. They create a âchain of thoughtâ before producing an answer, which looks a lot like introspection if you read it!
I can imagine how something like an LLM reasoning model could become conscious. Itâs interesting that they didnât use any reasoning models in that Turing test paper!
I donât find the Turing test evidence as convincing as you present it here.
Fair enough, I did not actually read the paper! I have talked to LLMs about consciousness and to me they seem pretty good at talking about it.
I agree that if each token you read is generated by a single forward pass through a network of fixed weights, then it seems hard to imagine how there could be any âinner lifeâ behind the words. There is no introspection. But this is not how the new generation of reasoning models work. They create a âchain of thoughtâ before producing an answer, which looks a lot like introspection if you read it!
The chain of thought is still generated via feed-forward next token prediction, right?
A commenter on my blog suggested that LLMs could still be doing enough internally that they are conscious even while generating only one token at a time, which sounds reasonable to me.
The chain of thought is still generated via feed-forward next token prediction, right?
Yes, it is.. But it still feels different to me.
If itâs possible to create consciousness on a computer at all, then at some level it will have to consist of mechanical operations which canât by themselves be conscious. This is because you could ultimately understand what it is doing as a set of simple instructions being carried out on a processor. So although I canât see how a single forward pass through a neural network could involve consciousness, I donât think a larger system being built out of these operations should rule out that larger system being conscious.
In a non-reasoning model, each token in the output is generated spontaneously, which means I canât see how there could be any conscious deliberation behind it. For example, it canât decide to spend longer thinking about a hard problem than an easier one, in the way a human might. I find it hard to get my head around a conscioussness that canât do that.
In a reasoning model, none of this applies.
(Although itâs true that the distinction probably isnât quite as clear cut as Iâm making out. A non-reasoning model could still decide to use its output to write out âchain of thoughtâ style reasoning, for example.)
Yes, it could well be that an LLM isnât conscious on a single pass, but it becomes conscious across multiple passes.
This is analogous to the Chinese room argument, but I donât take the Chinese room argument as a reductio ad absurdumâunless youâre a substance dualist or a panpsychist, I think you have to believe that a conscious being is made up of parts that are not themselves conscious.
(And even under panpsychism I think you still have to believe that the composed being is conscious in a way that the individual parts arenât? Not sure.)
I agree there is a non-negligible chance that existing LLMs are already conscious, and I think this is a really interesting and important discussion to have. Thanks for writing it up! I donât think I would put the chances as high as 10% though.
I donât find the Turing test evidence as convincing as you present it here. The paper you cited released their test online for people to try. I played it quite a lot, and I was always able to distinguish the human from the AI (they donât tell you which AI you are paired with, but presumably some of those were with GPT-4.5).
I think a kind of Turing test could be a good test for consciousness, but only if it is long, informed, and adversarial (e.g. as defined here: https://ââwww.metaculus.com/ââquestions/ââ11861/ââwhen-will-ai-pass-a-difficult-turing-test/ââ ). This version of the test has not been passed (although as someone pointed out to me on the forum before, this was not Turingâs original definition).
On the other hand, I donât find your strongest argument against LLM consciousness to be as convincing either. I agree that if each token you read is generated by a single forward pass through a network of fixed weights, then it seems hard to imagine how there could be any âinner lifeâ behind the words. There is no introspection. But this is not how the new generation of reasoning models work. They create a âchain of thoughtâ before producing an answer, which looks a lot like introspection if you read it!
I can imagine how something like an LLM reasoning model could become conscious. Itâs interesting that they didnât use any reasoning models in that Turing test paper!
Fair enough, I did not actually read the paper! I have talked to LLMs about consciousness and to me they seem pretty good at talking about it.
The chain of thought is still generated via feed-forward next token prediction, right?
A commenter on my blog suggested that LLMs could still be doing enough internally that they are conscious even while generating only one token at a time, which sounds reasonable to me.
Yes, it is.. But it still feels different to me.
If itâs possible to create consciousness on a computer at all, then at some level it will have to consist of mechanical operations which canât by themselves be conscious. This is because you could ultimately understand what it is doing as a set of simple instructions being carried out on a processor. So although I canât see how a single forward pass through a neural network could involve consciousness, I donât think a larger system being built out of these operations should rule out that larger system being conscious.
In a non-reasoning model, each token in the output is generated spontaneously, which means I canât see how there could be any conscious deliberation behind it. For example, it canât decide to spend longer thinking about a hard problem than an easier one, in the way a human might. I find it hard to get my head around a conscioussness that canât do that.
In a reasoning model, none of this applies.
(Although itâs true that the distinction probably isnât quite as clear cut as Iâm making out. A non-reasoning model could still decide to use its output to write out âchain of thoughtâ style reasoning, for example.)
Yes, it could well be that an LLM isnât conscious on a single pass, but it becomes conscious across multiple passes.
This is analogous to the Chinese room argument, but I donât take the Chinese room argument as a reductio ad absurdumâunless youâre a substance dualist or a panpsychist, I think you have to believe that a conscious being is made up of parts that are not themselves conscious.
(And even under panpsychism I think you still have to believe that the composed being is conscious in a way that the individual parts arenât? Not sure.)