My position statement (50% agree with the statement “If AGI goes well for humans, it’ll go well for animals”)
As a suffering-focused ethicist who generally rejects moral aggregation across individuals (I am most sympathetic to painism), I have a higher bar for “AGI going well for humans” for humans than many others do; it’s not clear to me that previous technological advances went well for humans
Agricultural revolution’s “luxury trap”: going from hunting-gathering to farming allowed humans to consolidate unprecedented wealth and power, but at the cost of the wellbeing/welfare/rights of very many humans
Perhaps similar arguments can be made for the industrial and digital revolutions
Even AGI Omelas is not an instance of AGI going well
“AGI going well” necessarily leaves many humans the stated preference to help animals (which might look like “abolishing animal exploitation and solving wild animal suffering”), and it certainly gives us the means and opportunity to do so
I happen to think that AGI going well for humans is unlikely, even by the lights of someone who is more upside-focused
We’re on track for creating something that is more intelligent than us (better at understanding the world and achieving goals within it) – and probably something with awareness, autonomy, agency, and the capacity for recursive self-improvement and self-replication – without understanding how it works, how to make it do what we want, or what it is we even want it to do
So, between normative and empirical claims, I believe a world in which AGI goes well for humans is a very small fraction of the possibility space
And when I try to think about what this AGI-going-well-for-humans world looks like, mostly I don’t really know, but it seems likely that in this world:
We retain and develop our moral wisdom (the most fundamental tenet of which is plausibly “non-maleficence and compassion towards all sentient beings”)
And we have the means to enact this moral wisdom
So, we abolish animal exploitation and solve wild animal suffering
Thus, AGI goes well for animals as well as humans!
Intense (and perhaps even moderate) human suffering eradicated
Probably, humans remaining empowered
Probably, our species isn’t disempowered by AGI; and
Probably, there isn’t severe inter-human inequality, specifically inequality of power; we don’t have a political elite determining how all other human lives go
Some kind of AGI technological innovation will be able to do 1); not clear to me how we get to 2), as we’ll probably need some kind of political pro-democracy innovation (I don’t think our existing political institutions will get us there).
What this world actually looks like, feels like, is very unclear to me! But if we do both those things, it seems more likely than not that we humans will both want and be able to help animals by abolishing animal exploitation and solving wild animal suffering.
I wonder though—would that kind of world, where humans are empowered but don’t experience intense (and perhaps moderate) suffering—be one where humans cared about animal welfare? I can see the intuition going either way. Either:
a) Extrapolating beyond person-to-person morality is (often) a luxury pursuit and more of it will happen in a post-scarcity world.
b) Caring about animal suffering in the food system and in nature requires compassion, and compassion is rooted in being able to imagine the states of the sufferer. If humans all live minimal suffering lives, they won’t be able to do so.
I need to think about b) more. I see arguments in both directions.
I don’t think I can properly imagine what it’s like to be tortured or eaten alive, and yet the thought of each happening to me or someone else makes me feel some combination of horror, fear, upset and compassion. And the idea of suffering more intense than torture or being eaten alive (if future artificially sentient beings have wider welfare ranges than we do) is terrifying to me.
But if I could never suffer worse than a pinprick, maybe I would stop caring about the most intense forms of suffering. Concerning stuff.
My position statement (50% agree with the statement “If AGI goes well for humans, it’ll go well for animals”)
As a suffering-focused ethicist who generally rejects moral aggregation across individuals (I am most sympathetic to painism), I have a higher bar for “AGI going well for humans” for humans than many others do; it’s not clear to me that previous technological advances went well for humans
Agricultural revolution’s “luxury trap”: going from hunting-gathering to farming allowed humans to consolidate unprecedented wealth and power, but at the cost of the wellbeing/welfare/rights of very many humans
Perhaps similar arguments can be made for the industrial and digital revolutions
Even AGI Omelas is not an instance of AGI going well
“AGI going well” necessarily leaves many humans the stated preference to help animals (which might look like “abolishing animal exploitation and solving wild animal suffering”), and it certainly gives us the means and opportunity to do so
I happen to think that AGI going well for humans is unlikely, even by the lights of someone who is more upside-focused
We’re on track for creating something that is more intelligent than us (better at understanding the world and achieving goals within it) – and probably something with awareness, autonomy, agency, and the capacity for recursive self-improvement and self-replication – without understanding how it works, how to make it do what we want, or what it is we even want it to do
So, between normative and empirical claims, I believe a world in which AGI goes well for humans is a very small fraction of the possibility space
And when I try to think about what this AGI-going-well-for-humans world looks like, mostly I don’t really know, but it seems likely that in this world:
We retain and develop our moral wisdom (the most fundamental tenet of which is plausibly “non-maleficence and compassion towards all sentient beings”)
And we have the means to enact this moral wisdom
So, we abolish animal exploitation and solve wild animal suffering
Thus, AGI goes well for animals as well as humans!
Can you say a bit more about what “AGI goes well for humans” means under your worldview? I hadn’t heard of painism.
I should have sketched this out more.
In my view, AGI going well for humans should see:
Intense (and perhaps even moderate) human suffering eradicated
Probably, humans remaining empowered
Probably, our species isn’t disempowered by AGI; and
Probably, there isn’t severe inter-human inequality, specifically inequality of power; we don’t have a political elite determining how all other human lives go
Some kind of AGI technological innovation will be able to do 1); not clear to me how we get to 2), as we’ll probably need some kind of political pro-democracy innovation (I don’t think our existing political institutions will get us there).
What this world actually looks like, feels like, is very unclear to me! But if we do both those things, it seems more likely than not that we humans will both want and be able to help animals by abolishing animal exploitation and solving wild animal suffering.
Thanks! That’s clarifying.
I wonder though—would that kind of world, where humans are empowered but don’t experience intense (and perhaps moderate) suffering—be one where humans cared about animal welfare? I can see the intuition going either way. Either:
a) Extrapolating beyond person-to-person morality is (often) a luxury pursuit and more of it will happen in a post-scarcity world.
b) Caring about animal suffering in the food system and in nature requires compassion, and compassion is rooted in being able to imagine the states of the sufferer. If humans all live minimal suffering lives, they won’t be able to do so.
I need to think about b) more. I see arguments in both directions.
I don’t think I can properly imagine what it’s like to be tortured or eaten alive, and yet the thought of each happening to me or someone else makes me feel some combination of horror, fear, upset and compassion. And the idea of suffering more intense than torture or being eaten alive (if future artificially sentient beings have wider welfare ranges than we do) is terrifying to me.
But if I could never suffer worse than a pinprick, maybe I would stop caring about the most intense forms of suffering. Concerning stuff.