Solving WAS intuitively seems too niche for people to deliberately change their mind on that, but I could be wrong. After all, the Bible says that the Lion will lie down with the lamb and eat straw like the ox, so it could be that human preferences tend to come back to the idea that animal suffering can be bad even when it doesn’t depend on human actions.
I guess the causal mechanism I’m thinking of here is:
Most humans feel at least a little sad when they see a baby gazelle being eaten alive by hyenas
AGI is so powerful that humans can order it to do things like “stop baby gazelles being eaten alive whilst retaining the beauty of nature and the complexity of ecosystems” and then it’ll just go away and do it somehow
Maybe this is foolish and naive on my part! And maybe I’m wrong to think our moral preferences/intuitions will be so robust to the disruption of AGI, even if AGI goes well for us.
Solving WAS intuitively seems too niche for people to deliberately change their mind on that, but I could be wrong. After all, the Bible says that the Lion will lie down with the lamb and eat straw like the ox, so it could be that human preferences tend to come back to the idea that animal suffering can be bad even when it doesn’t depend on human actions.
I guess the causal mechanism I’m thinking of here is:
Most humans feel at least a little sad when they see a baby gazelle being eaten alive by hyenas
AGI is so powerful that humans can order it to do things like “stop baby gazelles being eaten alive whilst retaining the beauty of nature and the complexity of ecosystems” and then it’ll just go away and do it somehow
Maybe this is foolish and naive on my part! And maybe I’m wrong to think our moral preferences/intuitions will be so robust to the disruption of AGI, even if AGI goes well for us.