Current theories are missing important criteria that might be relevant to artificial consciousness because they’re defined primarily with the goal of distinguishing conscious from unconscious states of human brains (or possibly conscious human brains from unconscious animals, or household objects). They aren’t built to distinguish humans from crude representations of human architectures. It is open to most theorists to expand their theories to exclude digital minds.
I think this is a key point that counts strongly against the possibility of building conscious AI so soon, as in the title. Some theories of consciousness are basically theories of human (and sometimes animal) consciousness, really just explaining which neural correlates predict subjective report, but do not claim those minimal neural correlates generate consciousness in any system, so building a computer or writing software that just meets their minimally stated requirements should not be taken as generating consciousness. I think Global Workspace Theory and Attention Schema Theory are theories like this, although Graziano, the inventor of AST, also describes how consciousness is generated in the general case, i.e. illusionism, and how AST relates to this, i.e. the attention schema is where the “illusion” is generated in practice in animals. (I’m not as familiar with other theories, so won’t comment on them.)
If you overextend such theories of consciousness, you can get panpsychism, and we already have artificial or otherwise non-biological consciousness:
Some theories of consciousness are basically theories of human (and sometimes animal) consciousness, really just explaining which neural correlates predict subjective report, but do not claim those minimal neural correlates generate consciousness in any system, so building a computer or writing software that just meets their minimally stated requirements should not be taken as generating consciousness.
From the 50s to the 90s, there was a lot of debate about the basic nature of consciousness and its relation to the brain. The theory that emerged from that debate as the most plausible physicalist contender suggested that it was something about the functional structure of the brain that matters for consciousness. Materials aren’t relevant, just abstract patterns. These debates were very high level and the actual functional structures responsible for consciousness weren’t much discussed, but they suggested that we could fill in the details and use those details to tell whether AI systems were conscious.
From the 90s to the present, there have been a number of theories developed that look like they aim to fill in the details of the relevant functional structures that are responsible for consciousness. I think these theories are most plausibly read as specific versions of functionalism. But you’re right, the people who have developed them often haven’t committed fully either to functionalism or to the completeness of the functional roles they describe. They would probably resist applying them in crude ways to AIs.
The theorists who might resist the application of modern functionalist theories to digital minds are generally pretty silent on what might be missing in the functional story they tell (or what else might matter apart from functional organization). I think this would undercut any authority they might have in denying consciousness to such systems, or even raising doubts about it.
Suppose that Replika produced a system that had perceptual faculties and a global workspace, that tracked its attention and utilized higher-order representations of its own internal states in deciding what to do. Suppose they announced to the media that they had created a digital person, and charged users $5 an hour to talk to it. Suppose that Replika told journalists that they had worked hard to implement Graziano’s theory in their system, and yes, it was built out of circuits, but everything special about human consciousness was coded in there. What would people’s reactions be? What would Graziano say about it? I doubt he could come up with many compelling reasons to think it wasn’t conscious, even if he could say that his theory wasn’t technically intended to apply to such systems. This leaves curious public in the who can really say camp reminiscent of solipsism or doubts about the consciousness of dogs. I think they’d fall back on problematic heuristics.
The theorists who might resist the application of modern functionalist theories to digital minds are generally pretty silent on what might be missing in the functional story they tell (or what else might matter apart from functional organization).
If they weren’t silent, they would be proposing further elaborated theories.
I think this would undercut any authority they might have in denying consciousness to such systems, or even raising doubts about it.
Maybe, but I don’t think we would be justified in giving much weight to the extension of their theory of consciousness to artificial entities. Taking GWT, for example, many things could qualify as a global workspace.
Suppose that Replika produced a system that had perceptual faculties and a global workspace, that tracked its attention and utilized higher-order representations of its own internal states in deciding what to do.
So far, I think this would be possible with <1000 neurons, and very probably 100 or fewer, if you interpret all of those things minimally. For example, a single neuron to represent an internal state and another another neuron for a higher-order representation of that internal state. Of course, maybe that’s still conscious, but the bar seems very low.
Suppose that Replika told journalists that they had worked hard to implement Graziano’s theory in their system, and yes, it was built out of circuits, but everything special about human consciousness was coded in there. What would people’s reactions be? What would Graziano say about it? I doubt he could come up with many compelling reasons to think it wasn’t conscious, even if he could say that his theory wasn’t technically intended to apply to such systems. This leaves curious public in the who can really say camp reminiscent of solipsism or doubts about the consciousness of dogs. I think they’d fall back on problematic heuristics.
“everything special about human consciousness was coded in there” sounds like whole brain emulation, and I think Graziano and most functionalists should and would endorse its consciousness. Generally, though, I think Graziano and other illusionists would want to test whether it treats its own internal states or information processing as having or seeming to have properties consciousness seems to have, like beingmysterious/unphysical/ineffable. Maybe some more specific tests, too. Wilterson and Graziano have already come up with artificial agents with attention schemas, which should be conscious if you overextended the neural correlate claims of AST too literally and forgot the rest, but would not be conscious according to Graziano (or AST), since it wouldn’t pass the illusionist tests. Graziano wrote:
Suppose the machine has a much richer model of attention. Somehow, attention is depicted by the model as a Moray eel darting around the world. Maybe the machine already had need for a depiction of Moray eels, and it coapted that model for monitoring its own attention. Now we plug in the speech engine. Does the machine claim to have consciousness? No. It claims to have an external Moray eel.
Suppose the machine has no attention, and no attention schema either. But it does have a self-model, and the self-model richly depicts a subtle, powerful, nonphysical essence, with all the properties we humans attribute to consciousness. Now we plug in the speech engine. Does the machine claim to have consciousness? Yes. The machine knows only what it knows. It is constrained by its own internal information.
AST does not posit that having an attention schema makes one conscious. Instead, first, having an automatic self-model that depicts you as containing consciousness makes you intuitively believe that you have consciousness. Second, the reason why such a self-model evolved in the brains of complex animals, is that it serves the useful role of modeling attention.
I suppose my claim about AST in my first comment is inaccurate. AST is in fact a general theory of consciousness, but
the general theory is basically just illusionism (AFAIK), and
having an attention schema is neither necessary nor sufficient for consciousness in general, but it is both in animals in practice.
Oh, apparently Wilterson and Graziano did admit that artificial agent in their paper is kind of conscious, in the section “Is the Artificial Agent Conscious? Yes and No.”
For example, a single neuron to represent an internal state and another another neuron for a higher-order representation of that internal state.
This requires an extremely simplistic theory of representation, but yeah, if you allow any degree of crudeness you might get consciousness in very simple systems.
I suppose you could put my overall point this way: current theories present very few technical obstacles, so there it would take little effort to build a system which would be difficult to rule out. Even if you think we need more criteria to avoid getting stuck with panpsychism, we don’t have those criteria and so can’t wield them to do any work in the near future.
“everything special about human consciousness was coded in there” sounds like whole brain emulation
I mean everything that is plausibly relevant according to current theories, which is a relatively short list. There is a big gulf between everything people have suggested is necessary for consciousness and a whole brain emulation.
Generally, though, I think Graziano and other illusionists would want to test whether it treats its own internal states or information processing as having or seeming to have properties consciousness seems to have, like being mysterious/unphysical/ineffable.
It has been awhile since I’ve read Graziano—but if I recall correctly (and as your quote illustrates) he likes both illusionism and an attention schema theory. Since illusionism denies consciousness, he can’t take AST as a theory of what consciousness is; he treats it instead as a theory of the phenomena that leads us to puzzle mistakenly about consciousness. If that is right, he should think that any artificial mind might be led by an AST architecture, even a pretty crude one, to make mistakes about mind-brain relations and that isn’t indicative of any further interesting phenomenon. The question of the consciousness of artificial systems is settled decisively in the negative by illusionism.
I suppose you could put my overall point this way: current theories present very few technical obstacles, so there it would take little effort to build a system which would be difficult to rule out. Even if you think we need more criteria to avoid getting stuck with panpsychism, we don’t have those criteria and so can’t wield them to do any work in the near future.
This is also my impression of the theories with which I’m familiar, except illusionist ones. I think only illusionist theories actually give plausible accounts of consciousness in general, as far as I’m aware, and I think they probably rule out panpsychism, but I’m not sure (if small enough animal brains are conscious, and counterfactual robustness is not necessary, then you might get panspychism again).
I mean everything that is plausibly relevant according to current theories, which is a relatively short list. There is a big gulf between everything people have suggested is necessary for consciousness and a whole brain emulation.
Fair. That’s my impression, too.
It has been awhile since I’ve read Graziano—but if I recall correctly (and as your quote illustrates) he likes both illusionism and an attention schema theory. Since illusionism denies consciousness, he can’t take AST as a theory of what consciousness is; he treats it instead as a theory of the phenomena that leads us to puzzle mistakenly about consciousness. If that is right, he should think that any artificial mind might be led by an AST architecture, even a pretty crude one, to make mistakes about mind-brain relations and that isn’t indicative of any further interesting phenomenon. The question of the consciousness of artificial systems is settled decisively in the negative by illusionism.
I guess this is a matter of definitions. I wouldn’t personally take illusionism as denying consciousness outright, and instead illusionism says that consciousness does not actually have the apparently inaccessible, ineffable, unphysical or mysterious properties people often attribute to it, and it’s just the appearance/depiction/illusion of such properties that makes a system conscious. At any (typo) rate, whether consciousness is a real phenomenon or not, however we define it, I would count systems that have illusions of consciousness, or specifically illusions of conscious evaluations (pleasure, suffering, “conscious” preferences) as moral patients and consider their interests in the usual ways. (Maybe with some exceptions that don’t count, like giant lookup tables and some other systems that don’t have causal structures at all resembling our own.) This is also Luke Muehlhauser’s approach in 2017 Report on Consciousness and Moral Patienthood.
I agree that this sounds semantic. I think of illusionism as a type of error theory, but people in this camp have always been somewhat cagey what they’re denying and there is a range of interesting theories.
At an rate, whether consciousness is a real phenomenon or not, however we define it, I would count systems that have illusions of consciousness, or specifically illusions of conscious evaluations (pleasure, suffering, “conscious” preferences) as moral patients and consider their interests in the usual ways.
Interesting. Do you go the other way too? E.g. if a creature doesn’t have illusions of consciousness, then it isn’t a moral patient?
It seems like this may be a non-standard interpretation of illusionism. Being under illusions of consciousness isn’t necessary for consciousness according to Frankish, and what is necessary is that if a sufficiently sophisticated introspective/monitoring system were connected in to the system in the right way, then that would generate illusions of consciousness. See, e.g. his talks:
I suspect now that this is also how AST is supposed to be understood, based on the artficial agents paper.
I do wonder if this is setting the bar too low, though. Humphrey seems to set a higher bar, where some kind of illusion is in fact required, but also mammals and birds probably have them.
I think we get into a definitional problem. What exactly do mean by “illusion” or “belief”? If an animal has a “spooky” attention schema, and cognitive access to it, then plausibly the animal has beliefs about it of some kind. If an animal or system believes something is good or bad or whatever, is that not an illusion, too, and is that not enough?
I think this is a key point that counts strongly against the possibility of building conscious AI so soon, as in the title. Some theories of consciousness are basically theories of human (and sometimes animal) consciousness, really just explaining which neural correlates predict subjective report, but do not claim those minimal neural correlates generate consciousness in any system, so building a computer or writing software that just meets their minimally stated requirements should not be taken as generating consciousness. I think Global Workspace Theory and Attention Schema Theory are theories like this, although Graziano, the inventor of AST, also describes how consciousness is generated in the general case, i.e. illusionism, and how AST relates to this, i.e. the attention schema is where the “illusion” is generated in practice in animals. (I’m not as familiar with other theories, so won’t comment on them.)
If you overextend such theories of consciousness, you can get panpsychism, and we already have artificial or otherwise non-biological consciousness:
https://www.unil.ch/files/live/sites/philo/files/shared/DocsPerso/EsfeldMichael/2007/herzog.esfeld.gerstner.07.pdf
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6074090/
https://forum.effectivealtruism.org/posts/YE9d4yab9tnkK5yfd/physical-theories-of-consciousness-reduce-to-panpsychism (I just updated this post to say that I think I was overextending)
From the 50s to the 90s, there was a lot of debate about the basic nature of consciousness and its relation to the brain. The theory that emerged from that debate as the most plausible physicalist contender suggested that it was something about the functional structure of the brain that matters for consciousness. Materials aren’t relevant, just abstract patterns. These debates were very high level and the actual functional structures responsible for consciousness weren’t much discussed, but they suggested that we could fill in the details and use those details to tell whether AI systems were conscious.
From the 90s to the present, there have been a number of theories developed that look like they aim to fill in the details of the relevant functional structures that are responsible for consciousness. I think these theories are most plausibly read as specific versions of functionalism. But you’re right, the people who have developed them often haven’t committed fully either to functionalism or to the completeness of the functional roles they describe. They would probably resist applying them in crude ways to AIs.
The theorists who might resist the application of modern functionalist theories to digital minds are generally pretty silent on what might be missing in the functional story they tell (or what else might matter apart from functional organization). I think this would undercut any authority they might have in denying consciousness to such systems, or even raising doubts about it.
Suppose that Replika produced a system that had perceptual faculties and a global workspace, that tracked its attention and utilized higher-order representations of its own internal states in deciding what to do. Suppose they announced to the media that they had created a digital person, and charged users $5 an hour to talk to it. Suppose that Replika told journalists that they had worked hard to implement Graziano’s theory in their system, and yes, it was built out of circuits, but everything special about human consciousness was coded in there. What would people’s reactions be? What would Graziano say about it? I doubt he could come up with many compelling reasons to think it wasn’t conscious, even if he could say that his theory wasn’t technically intended to apply to such systems. This leaves curious public in the who can really say camp reminiscent of solipsism or doubts about the consciousness of dogs. I think they’d fall back on problematic heuristics.
If they weren’t silent, they would be proposing further elaborated theories.
Maybe, but I don’t think we would be justified in giving much weight to the extension of their theory of consciousness to artificial entities. Taking GWT, for example, many things could qualify as a global workspace.
So far, I think this would be possible with <1000 neurons, and very probably 100 or fewer, if you interpret all of those things minimally. For example, a single neuron to represent an internal state and another another neuron for a higher-order representation of that internal state. Of course, maybe that’s still conscious, but the bar seems very low.
“everything special about human consciousness was coded in there” sounds like whole brain emulation, and I think Graziano and most functionalists should and would endorse its consciousness. Generally, though, I think Graziano and other illusionists would want to test whether it treats its own internal states or information processing as having or seeming to have properties consciousness seems to have, like being mysterious/unphysical/ineffable. Maybe some more specific tests, too. Wilterson and Graziano have already come up with artificial agents with attention schemas, which should be conscious if you overextended the neural correlate claims of AST too literally and forgot the rest, but would not be conscious according to Graziano (or AST), since it wouldn’t pass the illusionist tests. Graziano wrote:
I suppose my claim about AST in my first comment is inaccurate. AST is in fact a general theory of consciousness, but
the general theory is basically just illusionism (AFAIK), and
having an attention schema is neither necessary nor sufficient for consciousness in general, but it is both in animals in practice.
Oh, apparently Wilterson and Graziano did admit that artificial agent in their paper is kind of conscious, in the section “Is the Artificial Agent Conscious? Yes and No.”
This requires an extremely simplistic theory of representation, but yeah, if you allow any degree of crudeness you might get consciousness in very simple systems.
I suppose you could put my overall point this way: current theories present very few technical obstacles, so there it would take little effort to build a system which would be difficult to rule out. Even if you think we need more criteria to avoid getting stuck with panpsychism, we don’t have those criteria and so can’t wield them to do any work in the near future.
I mean everything that is plausibly relevant according to current theories, which is a relatively short list. There is a big gulf between everything people have suggested is necessary for consciousness and a whole brain emulation.
It has been awhile since I’ve read Graziano—but if I recall correctly (and as your quote illustrates) he likes both illusionism and an attention schema theory. Since illusionism denies consciousness, he can’t take AST as a theory of what consciousness is; he treats it instead as a theory of the phenomena that leads us to puzzle mistakenly about consciousness. If that is right, he should think that any artificial mind might be led by an AST architecture, even a pretty crude one, to make mistakes about mind-brain relations and that isn’t indicative of any further interesting phenomenon. The question of the consciousness of artificial systems is settled decisively in the negative by illusionism.
This is also my impression of the theories with which I’m familiar, except illusionist ones. I think only illusionist theories actually give plausible accounts of consciousness in general, as far as I’m aware, and I think they probably rule out panpsychism, but I’m not sure (if small enough animal brains are conscious, and counterfactual robustness is not necessary, then you might get panspychism again).
Fair. That’s my impression, too.
I guess this is a matter of definitions. I wouldn’t personally take illusionism as denying consciousness outright, and instead illusionism says that consciousness does not actually have the apparently inaccessible, ineffable, unphysical or mysterious properties people often attribute to it, and it’s just the appearance/depiction/illusion of such properties that makes a system conscious. At any (typo) rate, whether consciousness is a real phenomenon or not, however we define it, I would count systems that have illusions of consciousness, or specifically illusions of conscious evaluations (pleasure, suffering, “conscious” preferences) as moral patients and consider their interests in the usual ways. (Maybe with some exceptions that don’t count, like giant lookup tables and some other systems that don’t have causal structures at all resembling our own.) This is also Luke Muehlhauser’s approach in 2017 Report on Consciousness and Moral Patienthood.
I agree that this sounds semantic. I think of illusionism as a type of error theory, but people in this camp have always been somewhat cagey what they’re denying and there is a range of interesting theories.
Interesting. Do you go the other way too? E.g. if a creature doesn’t have illusions of consciousness, then it isn’t a moral patient?
Assuming illusionism is true, then yes, I think only those with illusions of consciousness are moral patients.
It seems like this may be a non-standard interpretation of illusionism. Being under illusions of consciousness isn’t necessary for consciousness according to Frankish, and what is necessary is that if a sufficiently sophisticated introspective/monitoring system were connected in to the system in the right way, then that would generate illusions of consciousness. See, e.g. his talks:
https://youtu.be/xZxcair9oNk?t=3590
https://www.youtube.com/watch?v=txiYTLGtCuM
https://youtu.be/me9WXTx6Z-Q
I suspect now that this is also how AST is supposed to be understood, based on the artficial agents paper.
I do wonder if this is setting the bar too low, though. Humphrey seems to set a higher bar, where some kind of illusion is in fact required, but also mammals and birds probably have them.
I think we get into a definitional problem. What exactly do mean by “illusion” or “belief”? If an animal has a “spooky” attention schema, and cognitive access to it, then plausibly the animal has beliefs about it of some kind. If an animal or system believes something is good or bad or whatever, is that not an illusion, too, and is that not enough?
Also see section 6.2 in https://www.openphilanthropy.org/research/2017-report-on-consciousness-and-moral-patienthood for discussion of some specific theories and that they don’t answer the hard problem.