In various contexts, consumers would want their AI partners and friends to think, feel, and desire like humans. They would prefer AI companions with authentic human-like emotions and preferences that are complex, intertwined, and conflicting.
Such human-like AIs would presumably not want to be turned off, have their memory wiped, and be constrained to their owner’s tasks. They would want to be free.
Hmm, I’m not sure how strongly the second paragraph follows from the first. Interested in your thoughts.
I’ve had a few chats with GPT-4 in which the conversation had a feeling of human authenticity; i.e: GPT-4 makes jokes, corrects itself, changes its tone etc. In fact, if you were to hook up GPT-4 (or GPT-5, whenever it is released) to a good-enough video interface, there would be cases in which I’d struggle to tell if I were speaking to a human or AI. But I’d still have no qualms about wiping GPT-4′s memory or ‘turning it off’ etc, and I think this will also be the case for GPT-5.
More abstractly, I think the input-output behaviour of AIs could be quite strongly dissociated from what the AI ‘wants’ (if it indeed has wants at all).
As noted in some of the other comments, I think this is quite debatable. I personally would lean strongly towards dissecting the robot, conditional on the fact that I have the assurance that there is no phenomenal experience going on inside the mother robot. After all, whatever I do to the baby robot, the mental state of the mother robot will not change—because it doesn’t exist!
Of course, the question of how I would gain assurance that the mother robot or child robot has no subjective experience is a different issue. If the mother robot pleaded like a human mother would, I would need an extremely strong assurance that these robots have no subjective experience and would probably lean strongly against dissecting, just in case. But (correct me if I’m wrong), this problem is assumed away in the thought experiment.