I’m not sure what you’re imagining here. If you give people a trolley problem (only via text) and say on one track, there’s a dog and on the other one, there’s a computer program Eliza and they can chat to either, most would choose to save the dog, even if its only text output were “whoof whoof”.
What I’m imagining, which I evidently didn’t make clear enough, is not a trolley problem but simply trying to discern whether something else is conscious without knowing whether it has “faces, limbs, fur and a squishy body”, such as reading its output [if any] over a remote computer terminal.[1] In these circumstances, not only will humans be unable to find any grounds for empathy with almost all sentient beings, but they will find plenty of grounds to empathise with or at least attribute motivation and intent to software programs.[2] So in the absence of context there’s definitely a bias in mind attribution towards symbol manipulators; even trivially simple ones that merely mimic or perform arithmetic.
On the other hand it seems like the “faces, limbs, fur and a squishy body” are actually a relatively useful heuristic, especially since adults are seldom deceived by taxidermy or cuddly toys in comparison to how easily they’re impressed by “cheap parlour trick” level AI
Are people more likely to empathise with entities with “faces, limbs, fur and a squishy body” than disembodied entities with apparent facility with symbol manipulation? Possibly, though I think this varies,[3] but the relevant question is: are people more likely misplace their empathy in imputing basic consciousness to other mammals or imputing heightened consciousness to anything that can beat them at chess
The level of misattribution matters too. Our sense of empathy anthropomorphises dogs by overestimating their grasp of language and underestimating the extent to which they are motivated by smell, and anthropomorphises irritating repetitive hardcoded chatbots by assigning meaning and motivation which simply doesn’t exist.
Most non-dualists would say consciousness is a feature of information processing (functionalists, illusionists, non-reductive materialists) or something as fundamental as physics (Russelian monism, pan(proto)psychism)...The phrase “rooted in [biochemical processes]” is the least controversial but it still connotes something most might not endorse—i.e. that biology and chemistry is the correct category or level of description
The rooted in biochemical processes is the bit I’m aiming for here; I am not aware of a non-dualist theory which roots human cognition in something other than the biochemical processes of the body (I don’t think the biochemical processes of the body themselves particularly care whether philosophers of mind label them as the consequence of evolutionary imperatives, physics, function, or illusion.)[4] Perhaps I can only be fully confident of my own consciousness, but its relationship with my biochemistry and physiology does at least comes with a bunch of hypotheses originally tested on similar organisms (albeit many of those hypotheses I’d rather not test …).
Like Turing’s eponymous test, only not explicitly a test. Eliza might convince a human not primed to look for evidence it’s just a shoddy computer program that its outputs represent a stream of conscious thought; nobody’s going to try to follow the strains of thought in a dog’s typing...
or indeed if between 93% and 99.9% of them live lifestyles too chaste or avant garde to acknowledge the possibility that changes in their hormone balance and neurological state associated with [the prospect of] sex might be linked to evolutionary imperatives to reproduce ;-)
I can experimentally verify claims made about how certain changes to my biochemistry or physiology would affect my consciousness, though in most cases I’d rather not :)
Thanks for clarifying—sorry it might sound like I was twisting your words—I was trying to think through multiple versions of the experiment you propose.
The amount to which we attribute/misattribute consciousness to different entities depends on the correct theory, so it is very uncertain at this point. But I would endorse this broader research program of systematically decoding which of our intuitions about consciousness are biases and which are valid measurements of brain data.
One reason why I thought about Trolley problems was that they show not only % of people who have an abstract belief about consciousness but also the degree / intensity of its perceived experiences. I’m surprised to see a significant fraction of people (1, 2) say current AI is conscious, although a poll about a personal sacrifice like this one (in a less narrow Twitter bubble) might be more relevant to assess how serious they are—and might better model the kind of moral error that we’re more likely to make during the AGI transformation.
Regarding “biochemical processes”—the phrasing matters a lot here. Searle, who came up with The Chinese Room, concludes this thought experiment by suggesting thinking requires the specific biochemistry that brains use just like lactation or photosynthesis are defined by specific molecules, rather than algorithms. This formulation is specifically chosen in contrast to functionalist/computationalist views which are mainstream nowadays.
What I’m imagining, which I evidently didn’t make clear enough, is not a trolley problem but simply trying to discern whether something else is conscious without knowing whether it has “faces, limbs, fur and a squishy body”, such as reading its output [if any] over a remote computer terminal.[1] In these circumstances, not only will humans be unable to find any grounds for empathy with almost all sentient beings, but they will find plenty of grounds to empathise with or at least attribute motivation and intent to software programs.[2] So in the absence of context there’s definitely a bias in mind attribution towards symbol manipulators; even trivially simple ones that merely mimic or perform arithmetic.
On the other hand it seems like the “faces, limbs, fur and a squishy body” are actually a relatively useful heuristic, especially since adults are seldom deceived by taxidermy or cuddly toys in comparison to how easily they’re impressed by “cheap parlour trick” level AI
Are people more likely to empathise with entities with “faces, limbs, fur and a squishy body” than disembodied entities with apparent facility with symbol manipulation? Possibly, though I think this varies,[3] but the relevant question is: are people more likely misplace their empathy in imputing basic consciousness to other mammals or imputing heightened consciousness to anything that can beat them at chess
The level of misattribution matters too. Our sense of empathy anthropomorphises dogs by overestimating their grasp of language and underestimating the extent to which they are motivated by smell, and anthropomorphises irritating repetitive hardcoded chatbots by assigning meaning and motivation which simply doesn’t exist.
The rooted in biochemical processes is the bit I’m aiming for here; I am not aware of a non-dualist theory which roots human cognition in something other than the biochemical processes of the body (I don’t think the biochemical processes of the body themselves particularly care whether philosophers of mind label them as the consequence of evolutionary imperatives, physics, function, or illusion.)[4] Perhaps I can only be fully confident of my own consciousness, but its relationship with my biochemistry and physiology does at least comes with a bunch of hypotheses originally tested on similar organisms (albeit many of those hypotheses I’d rather not test …).
Like Turing’s eponymous test, only not explicitly a test. Eliza might convince a human not primed to look for evidence it’s just a shoddy computer program that its outputs represent a stream of conscious thought; nobody’s going to try to follow the strains of thought in a dog’s typing...
including those [near]-universally agreed to be too simple to have any sort of motivation, intent or consciousness
No shortage of people who have developed feelings for ChatGPT, and I bet most of them eat cute farm animals
or indeed if between 93% and 99.9% of them live lifestyles too chaste or avant garde to acknowledge the possibility that changes in their hormone balance and neurological state associated with [the prospect of] sex might be linked to evolutionary imperatives to reproduce ;-)
I can experimentally verify claims made about how certain changes to my biochemistry or physiology would affect my consciousness, though in most cases I’d rather not :)
Thanks for clarifying—sorry it might sound like I was twisting your words—I was trying to think through multiple versions of the experiment you propose.
The amount to which we attribute/misattribute consciousness to different entities depends on the correct theory, so it is very uncertain at this point. But I would endorse this broader research program of systematically decoding which of our intuitions about consciousness are biases and which are valid measurements of brain data.
One reason why I thought about Trolley problems was that they show not only % of people who have an abstract belief about consciousness but also the degree / intensity of its perceived experiences. I’m surprised to see a significant fraction of people (1, 2) say current AI is conscious, although a poll about a personal sacrifice like this one (in a less narrow Twitter bubble) might be more relevant to assess how serious they are—and might better model the kind of moral error that we’re more likely to make during the AGI transformation.
Regarding “biochemical processes”—the phrasing matters a lot here. Searle, who came up with The Chinese Room, concludes this thought experiment by suggesting thinking requires the specific biochemistry that brains use just like lactation or photosynthesis are defined by specific molecules, rather than algorithms. This formulation is specifically chosen in contrast to functionalist/computationalist views which are mainstream nowadays.