Strongly agree that if lock-in happens, it will be very important for those controlling the AIs to care about all sentient beings. My impression of top AGI researchers is that most take AI sentience pretty seriously as a possibility, and it seems hard for someone to think this without also believing animals can be sentient.
Obviously this is less true the further you get from AI safety/OpenAI/DeepMind/Anthropic. An important question is, if AGI happens and the control problem is solved, who ends up deciding what the AGI values?
I’m pretty uncomfortable with the idea of random computer scientists, tech moguls, or politicians having all the power. Seems like the ideal to aim for is a democratic process structured to represent the reflective interests of all sentient beings. But this would be extremely difficult to do in practice. Realistically I expect a messy power struggle between various interest groups. In that case, outreach to leaders of all the interest groups to protect nonhuman minds is crucial, as you suggest.
My impression of top AGI researchers is that most take AI sentience pretty seriously as a possibility, and it seems hard for someone to think this without also believing animals can be sentient.
Strongly agree that if lock-in happens, it will be very important for those controlling the AIs to care about all sentient beings. My impression of top AGI researchers is that most take AI sentience pretty seriously as a possibility, and it seems hard for someone to think this without also believing animals can be sentient.
Obviously this is less true the further you get from AI safety/OpenAI/DeepMind/Anthropic. An important question is, if AGI happens and the control problem is solved, who ends up deciding what the AGI values?
I’m pretty uncomfortable with the idea of random computer scientists, tech moguls, or politicians having all the power. Seems like the ideal to aim for is a democratic process structured to represent the reflective interests of all sentient beings. But this would be extremely difficult to do in practice. Realistically I expect a messy power struggle between various interest groups. In that case, outreach to leaders of all the interest groups to protect nonhuman minds is crucial, as you suggest.
I wrote some related thoughts here, curious what you think.
I am not saying this is common, but it is alarming that Eliezer Yudkowsky, a pretty prominent figure in the space, thinks that AI sentience is possible but nonhuman animals are not sentient.
Agreed, it’s a pretty bizarre take. I’d be curious whether his views have changed since he wrote that FB post
Also, Holden Karnofsky (not so confidently) think that humans matter astronomically more than nonhuman animals. At the same time he thinks that digital people is possible.