We are evolved to feel empathy—that is, modelling “what it is like to be them”—towards entities that have faces, limbs, fur and a squishy body. We feel a lot of empathy for pets and babies—entities that don’t control language
People empathise with ChatGPT transcripts too; the key distinction is between empathising with a wide range of physiological and behavioural cues as opposed to a narrow one. For all its flaws, observing whether they are like us seems like a more plausible way of establishing likely consciousness than aptitude for symbolic manipulation. Disintermediated by a computer to replace biases introduced by cuteness and non-verbal expressiveness with biases introduced by symbolic manipulation, humans would without exception rate a pocket calculator or Eliza- style toy script as more likely to be conscious than a dog or a two year old child. I don’t think anyone sincerely believes this to actually be the case. A corollary of this is that facility with human language and mathematics—something most entities considered to be conscious do not possess—is not a particularly good standalone proxy for consciousness, even if the entity under examination is much much better at it than Eliza.
Unless we’re positing dualism, what we perceive at consciousness is an emergent property of complex chemical processes rooted in our biology (and the imperatives of our biology to survive and self replicate.. That’s the case whether we empathise with other entities that share this biology, dispassionately analyse the likelihood we evolved from a common ancestor or torture them into demonstrating similar stress hormone responses to humans. That doesn’t necessarily mean warm blooded DNA-replicating machines with limbs and fur and cute eyes are the only possible form of consciousness, but it is something we have and current “AI” doesn’t even have a loose analogue of. And yes, obsession with symbolic manipulation is doing an awful lot of work to explain why people are concerned about the consciousness of assemblies of silicon chips running specific software whilst disregarding the possibility of the sentience of more complex and interesting long running processes like forests, rivers or planetary systems, or indeed larger assemblies of silicon chips running software that doesn’t transform human text into cute replies.
Disintermediated by a computer to replace biases introduced by cuteness and non-verbal expressiveness with biases introduced by symbolic manipulation, humans would without exception rate a pocket calculator or Eliza- style toy script as more likely to be conscious than a dog or a two year old child. I don’t think anyone sincerely believes this to actually be the case.
I’m not sure what you’re imagining here. If you give people a trolley problem (only via text) and say on one track, there’s a dog and on the other one, there’s a computer program Eliza and they can chat to either, most would choose to save the dog, even if its only text output were “whoof whoof”.
If you’re imagining the thought experiment would somehow block them to make the inference that one entity is an actual dog and the other a program, then yes: I agree with the point that language increases empathy but I’d say the magnitude is much smaller than “non-verbal cues”. If you had a Trolley dilemma with one blank track and one track with either a dog or Eliza, I think 90% would pay $0 to save Eliza but a often a lot of money to save the dog.
Unless we’re positing dualism, what we perceive at consciousness is an emergent property of complex chemical processes rooted in our biology (and the imperatives of our biology to survive and self replicate.
Most non-dualists would say consciousness is a feature of information processing (functionalists, illusionists, non-reductive materialists) or something as fundamental as physics (Russelian monism, pan(proto)psychism). The particular emergentist and biological theory that is rooted in the instinct to self-replicate and survive is something I’d expect 0.1-7% of philosophers of mind to endorse. But whatever the actual percentages, I definitely disagree dualism and this theory are the only options. The phrase “rooted in [biochemical processes]” is the least controversial but it still connotes something most might not endorse—i.e. that biology and chemistry is the correct category or level of description (Axis 3 in this taxonomy).
I’m not sure what you’re imagining here. If you give people a trolley problem (only via text) and say on one track, there’s a dog and on the other one, there’s a computer program Eliza and they can chat to either, most would choose to save the dog, even if its only text output were “whoof whoof”.
What I’m imagining, which I evidently didn’t make clear enough, is not a trolley problem but simply trying to discern whether something else is conscious without knowing whether it has “faces, limbs, fur and a squishy body”, such as reading its output [if any] over a remote computer terminal.[1] In these circumstances, not only will humans be unable to find any grounds for empathy with almost all sentient beings, but they will find plenty of grounds to empathise with or at least attribute motivation and intent to software programs.[2] So in the absence of context there’s definitely a bias in mind attribution towards symbol manipulators; even trivially simple ones that merely mimic or perform arithmetic.
On the other hand it seems like the “faces, limbs, fur and a squishy body” are actually a relatively useful heuristic, especially since adults are seldom deceived by taxidermy or cuddly toys in comparison to how easily they’re impressed by “cheap parlour trick” level AI
Are people more likely to empathise with entities with “faces, limbs, fur and a squishy body” than disembodied entities with apparent facility with symbol manipulation? Possibly, though I think this varies,[3] but the relevant question is: are people more likely misplace their empathy in imputing basic consciousness to other mammals or imputing heightened consciousness to anything that can beat them at chess
The level of misattribution matters too. Our sense of empathy anthropomorphises dogs by overestimating their grasp of language and underestimating the extent to which they are motivated by smell, and anthropomorphises irritating repetitive hardcoded chatbots by assigning meaning and motivation which simply doesn’t exist.
Most non-dualists would say consciousness is a feature of information processing (functionalists, illusionists, non-reductive materialists) or something as fundamental as physics (Russelian monism, pan(proto)psychism)...The phrase “rooted in [biochemical processes]” is the least controversial but it still connotes something most might not endorse—i.e. that biology and chemistry is the correct category or level of description
The rooted in biochemical processes is the bit I’m aiming for here; I am not aware of a non-dualist theory which roots human cognition in something other than the biochemical processes of the body (I don’t think the biochemical processes of the body themselves particularly care whether philosophers of mind label them as the consequence of evolutionary imperatives, physics, function, or illusion.)[4] Perhaps I can only be fully confident of my own consciousness, but its relationship with my biochemistry and physiology does at least comes with a bunch of hypotheses originally tested on similar organisms (albeit many of those hypotheses I’d rather not test …).
Like Turing’s eponymous test, only not explicitly a test. Eliza might convince a human not primed to look for evidence it’s just a shoddy computer program that its outputs represent a stream of conscious thought; nobody’s going to try to follow the strains of thought in a dog’s typing...
or indeed if between 93% and 99.9% of them live lifestyles too chaste or avant garde to acknowledge the possibility that changes in their hormone balance and neurological state associated with [the prospect of] sex might be linked to evolutionary imperatives to reproduce ;-)
I can experimentally verify claims made about how certain changes to my biochemistry or physiology would affect my consciousness, though in most cases I’d rather not :)
Thanks for clarifying—sorry it might sound like I was twisting your words—I was trying to think through multiple versions of the experiment you propose.
The amount to which we attribute/misattribute consciousness to different entities depends on the correct theory, so it is very uncertain at this point. But I would endorse this broader research program of systematically decoding which of our intuitions about consciousness are biases and which are valid measurements of brain data.
One reason why I thought about Trolley problems was that they show not only % of people who have an abstract belief about consciousness but also the degree / intensity of its perceived experiences. I’m surprised to see a significant fraction of people (1, 2) say current AI is conscious, although a poll about a personal sacrifice like this one (in a less narrow Twitter bubble) might be more relevant to assess how serious they are—and might better model the kind of moral error that we’re more likely to make during the AGI transformation.
Regarding “biochemical processes”—the phrasing matters a lot here. Searle, who came up with The Chinese Room, concludes this thought experiment by suggesting thinking requires the specific biochemistry that brains use just like lactation or photosynthesis are defined by specific molecules, rather than algorithms. This formulation is specifically chosen in contrast to functionalist/computationalist views which are mainstream nowadays.
People empathise with ChatGPT transcripts too; the key distinction is between empathising with a wide range of physiological and behavioural cues as opposed to a narrow one. For all its flaws, observing whether they are like us seems like a more plausible way of establishing likely consciousness than aptitude for symbolic manipulation. Disintermediated by a computer to replace biases introduced by cuteness and non-verbal expressiveness with biases introduced by symbolic manipulation, humans would without exception rate a pocket calculator or Eliza- style toy script as more likely to be conscious than a dog or a two year old child. I don’t think anyone sincerely believes this to actually be the case. A corollary of this is that facility with human language and mathematics—something most entities considered to be conscious do not possess—is not a particularly good standalone proxy for consciousness, even if the entity under examination is much much better at it than Eliza.
Unless we’re positing dualism, what we perceive at consciousness is an emergent property of complex chemical processes rooted in our biology (and the imperatives of our biology to survive and self replicate.. That’s the case whether we empathise with other entities that share this biology, dispassionately analyse the likelihood we evolved from a common ancestor or torture them into demonstrating similar stress hormone responses to humans. That doesn’t necessarily mean warm blooded DNA-replicating machines with limbs and fur and cute eyes are the only possible form of consciousness, but it is something we have and current “AI” doesn’t even have a loose analogue of. And yes, obsession with symbolic manipulation is doing an awful lot of work to explain why people are concerned about the consciousness of assemblies of silicon chips running specific software whilst disregarding the possibility of the sentience of more complex and interesting long running processes like forests, rivers or planetary systems, or indeed larger assemblies of silicon chips running software that doesn’t transform human text into cute replies.
I’m not sure what you’re imagining here. If you give people a trolley problem (only via text) and say on one track, there’s a dog and on the other one, there’s a computer program Eliza and they can chat to either, most would choose to save the dog, even if its only text output were “whoof whoof”.
If you’re imagining the thought experiment would somehow block them to make the inference that one entity is an actual dog and the other a program, then yes: I agree with the point that language increases empathy but I’d say the magnitude is much smaller than “non-verbal cues”. If you had a Trolley dilemma with one blank track and one track with either a dog or Eliza, I think 90% would pay $0 to save Eliza but a often a lot of money to save the dog.
Most non-dualists would say consciousness is a feature of information processing (functionalists, illusionists, non-reductive materialists) or something as fundamental as physics (Russelian monism, pan(proto)psychism). The particular emergentist and biological theory that is rooted in the instinct to self-replicate and survive is something I’d expect 0.1-7% of philosophers of mind to endorse. But whatever the actual percentages, I definitely disagree dualism and this theory are the only options. The phrase “rooted in [biochemical processes]” is the least controversial but it still connotes something most might not endorse—i.e. that biology and chemistry is the correct category or level of description (Axis 3 in this taxonomy).
What I’m imagining, which I evidently didn’t make clear enough, is not a trolley problem but simply trying to discern whether something else is conscious without knowing whether it has “faces, limbs, fur and a squishy body”, such as reading its output [if any] over a remote computer terminal.[1] In these circumstances, not only will humans be unable to find any grounds for empathy with almost all sentient beings, but they will find plenty of grounds to empathise with or at least attribute motivation and intent to software programs.[2] So in the absence of context there’s definitely a bias in mind attribution towards symbol manipulators; even trivially simple ones that merely mimic or perform arithmetic.
On the other hand it seems like the “faces, limbs, fur and a squishy body” are actually a relatively useful heuristic, especially since adults are seldom deceived by taxidermy or cuddly toys in comparison to how easily they’re impressed by “cheap parlour trick” level AI
Are people more likely to empathise with entities with “faces, limbs, fur and a squishy body” than disembodied entities with apparent facility with symbol manipulation? Possibly, though I think this varies,[3] but the relevant question is: are people more likely misplace their empathy in imputing basic consciousness to other mammals or imputing heightened consciousness to anything that can beat them at chess
The level of misattribution matters too. Our sense of empathy anthropomorphises dogs by overestimating their grasp of language and underestimating the extent to which they are motivated by smell, and anthropomorphises irritating repetitive hardcoded chatbots by assigning meaning and motivation which simply doesn’t exist.
The rooted in biochemical processes is the bit I’m aiming for here; I am not aware of a non-dualist theory which roots human cognition in something other than the biochemical processes of the body (I don’t think the biochemical processes of the body themselves particularly care whether philosophers of mind label them as the consequence of evolutionary imperatives, physics, function, or illusion.)[4] Perhaps I can only be fully confident of my own consciousness, but its relationship with my biochemistry and physiology does at least comes with a bunch of hypotheses originally tested on similar organisms (albeit many of those hypotheses I’d rather not test …).
Like Turing’s eponymous test, only not explicitly a test. Eliza might convince a human not primed to look for evidence it’s just a shoddy computer program that its outputs represent a stream of conscious thought; nobody’s going to try to follow the strains of thought in a dog’s typing...
including those [near]-universally agreed to be too simple to have any sort of motivation, intent or consciousness
No shortage of people who have developed feelings for ChatGPT, and I bet most of them eat cute farm animals
or indeed if between 93% and 99.9% of them live lifestyles too chaste or avant garde to acknowledge the possibility that changes in their hormone balance and neurological state associated with [the prospect of] sex might be linked to evolutionary imperatives to reproduce ;-)
I can experimentally verify claims made about how certain changes to my biochemistry or physiology would affect my consciousness, though in most cases I’d rather not :)
Thanks for clarifying—sorry it might sound like I was twisting your words—I was trying to think through multiple versions of the experiment you propose.
The amount to which we attribute/misattribute consciousness to different entities depends on the correct theory, so it is very uncertain at this point. But I would endorse this broader research program of systematically decoding which of our intuitions about consciousness are biases and which are valid measurements of brain data.
One reason why I thought about Trolley problems was that they show not only % of people who have an abstract belief about consciousness but also the degree / intensity of its perceived experiences. I’m surprised to see a significant fraction of people (1, 2) say current AI is conscious, although a poll about a personal sacrifice like this one (in a less narrow Twitter bubble) might be more relevant to assess how serious they are—and might better model the kind of moral error that we’re more likely to make during the AGI transformation.
Regarding “biochemical processes”—the phrasing matters a lot here. Searle, who came up with The Chinese Room, concludes this thought experiment by suggesting thinking requires the specific biochemistry that brains use just like lactation or photosynthesis are defined by specific molecules, rather than algorithms. This formulation is specifically chosen in contrast to functionalist/computationalist views which are mainstream nowadays.