This is definitely a hard debate to disentangle, because I would personally reject the question of alignment as a crux. For now, I strongly believe that the total welfare of animals has been entirely uncorrelated with our moral intentions toward animals. Total welfare has mostly changed because of land use, due to human interests.
I agree that in AGI-transformed futures that go well for humans, human desires may start playing a larger role. However, I expect that whether we mean well for animals (or don’t care much about them) will not be cleanly correlated with outcomes for them.
There are worlds where we mean well for a large part of animals, stop intentionally killing them, and help certain wild animals. But that world could very well end up having a large population of animals living bad lives.
On the other hand, out of apathy and even negative feeling toward wild animals, we may decide to limit their spread and use resources in a way that optimizes for human flourishing, over animal abundance. That world could end up being much better for animal welfare.
Maybe some extreme scenarios tip the scales, for example if we bred incredibly happy genetically modified animals due to positive feelings toward them. But I’m not confident on putting any weight on such utilitarian-leaning scenarios when assessing post-AGI futures. Because part of the reason human moral intentions are not correlated with total animal welfare is that humans are not scope-sensitive utilitarians.
What kinds of values will humans have post-AGI, if AGI goes well for us? We don’t need to be scope-sensitive utilitarians to want to adopt even radical preferences like ending animal exploitation and solving WAS, no? (Most humans don’t like factory farming or the idea of cute animals being eaten alive.)
Solving WAS intuitively seems too niche for people to deliberately change their mind on that, but I could be wrong. After all, the Bible says that the Lion will lie down with the lamb and eat straw like the ox, so it could be that human preferences tend to come back to the idea that animal suffering can be bad even when it doesn’t depend on human actions.
I guess the causal mechanism I’m thinking of here is:
Most humans feel at least a little sad when they see a baby gazelle being eaten alive by hyenas
AGI is so powerful that humans can order it to do things like “stop baby gazelles being eaten alive whilst retaining the beauty of nature and the complexity of ecosystems” and then it’ll just go away and do it somehow
Maybe this is foolish and naive on my part! And maybe I’m wrong to think our moral preferences/intuitions will be so robust to the disruption of AGI, even if AGI goes well for us.
This is definitely a hard debate to disentangle, because I would personally reject the question of alignment as a crux. For now, I strongly believe that the total welfare of animals has been entirely uncorrelated with our moral intentions toward animals. Total welfare has mostly changed because of land use, due to human interests.
I agree that in AGI-transformed futures that go well for humans, human desires may start playing a larger role. However, I expect that whether we mean well for animals (or don’t care much about them) will not be cleanly correlated with outcomes for them.
There are worlds where we mean well for a large part of animals, stop intentionally killing them, and help certain wild animals. But that world could very well end up having a large population of animals living bad lives.
On the other hand, out of apathy and even negative feeling toward wild animals, we may decide to limit their spread and use resources in a way that optimizes for human flourishing, over animal abundance. That world could end up being much better for animal welfare.
Maybe some extreme scenarios tip the scales, for example if we bred incredibly happy genetically modified animals due to positive feelings toward them. But I’m not confident on putting any weight on such utilitarian-leaning scenarios when assessing post-AGI futures. Because part of the reason human moral intentions are not correlated with total animal welfare is that humans are not scope-sensitive utilitarians.
What kinds of values will humans have post-AGI, if AGI goes well for us? We don’t need to be scope-sensitive utilitarians to want to adopt even radical preferences like ending animal exploitation and solving WAS, no? (Most humans don’t like factory farming or the idea of cute animals being eaten alive.)
Solving WAS intuitively seems too niche for people to deliberately change their mind on that, but I could be wrong. After all, the Bible says that the Lion will lie down with the lamb and eat straw like the ox, so it could be that human preferences tend to come back to the idea that animal suffering can be bad even when it doesn’t depend on human actions.
I guess the causal mechanism I’m thinking of here is:
Most humans feel at least a little sad when they see a baby gazelle being eaten alive by hyenas
AGI is so powerful that humans can order it to do things like “stop baby gazelles being eaten alive whilst retaining the beauty of nature and the complexity of ecosystems” and then it’ll just go away and do it somehow
Maybe this is foolish and naive on my part! And maybe I’m wrong to think our moral preferences/intuitions will be so robust to the disruption of AGI, even if AGI goes well for us.