+1. I appreciated @RobertM’sarticulation of this problem for animal welfare in particular:
I think the interventions for ensuring that animal welfare is good after we hit transformative AI probably look very different from interventions in the pretty small slice of worlds where the world looks very boring in a few decades.
…
If we achieve transformative AI and then don’t all die (because we solved alignment), then I don’t think the world will continue to have an “agricultural industry” in any meaningful sense (or, really, any other traditional industry; strong nanotech seems like it ought to let you solve for nearly everything else). Even if the economics and sociology work out such that some people will want to continue farming real animals instead of enjoying the much cheaper cultured meat of vastly superior quality, there will be approximately nobody interested in ensuring those animals are suffering, and the cost for ensuring that they don’t suffer will be trivial.
[...] if you think it’s at all plausible that we achieve TAI in a way that locks in reflectively-unendorsed values which lead to huge quantities of animal suffering, that seems like it ought to dominate effectively all other considerations in terms of interventions w.r.t. future animal welfare.
I’ve actually tried asking/questioning a few animal welfare folks for their takes here, but I’ve yet to hear back anything that sounded compelling (to me). (If anyone reading this has an argument for why ‘standard’ animal welfare interventions are robust to the above, then I’d love to hear it!)
Animal welfare guy tuning in. My own take is that the majority of the world actually is almost entirely indifferent about animal suffering, so if AI tries to reflect global values (not just the values of the progressive, elite silicon valley bubble) there is a real risk that it will be indifferent to animal suffering. Consider how Foie Gras is still legal in most countries, or bullfighting, both of which are totally unnecessary. And those are just examples from western countries.
I think it’s very likely that TAI will lock in only a very mild concern for animal welfare. Or perhaps, concern for animal welfare in certain contexts (e.g. pets), and none in others (e.g. chicken). Maybe that will lead to a future without factory farming, but it will lead to a future with unnecessary animal suffering nonetheless.
What I’m not sure about is: how do we ensure that TAI locks in a strong valuation of animal welfare? One route is to try to change how much society cares about animal welfare, and hope that TAI then reflects that. I guess this is the hope of many animal advocates. But I admit that seems too slow to work at this stage, so I agree that animal advocates should probably prioritize trying to influence those developing AI right now.
+1. I appreciated @RobertM’s articulation of this problem for animal welfare in particular:
I’ve actually tried asking/questioning a few animal welfare folks for their takes here, but I’ve yet to hear back anything that sounded compelling (to me). (If anyone reading this has an argument for why ‘standard’ animal welfare interventions are robust to the above, then I’d love to hear it!)
Animal welfare guy tuning in. My own take is that the majority of the world actually is almost entirely indifferent about animal suffering, so if AI tries to reflect global values (not just the values of the progressive, elite silicon valley bubble) there is a real risk that it will be indifferent to animal suffering. Consider how Foie Gras is still legal in most countries, or bullfighting, both of which are totally unnecessary. And those are just examples from western countries.
I think it’s very likely that TAI will lock in only a very mild concern for animal welfare. Or perhaps, concern for animal welfare in certain contexts (e.g. pets), and none in others (e.g. chicken). Maybe that will lead to a future without factory farming, but it will lead to a future with unnecessary animal suffering nonetheless.
What I’m not sure about is: how do we ensure that TAI locks in a strong valuation of animal welfare? One route is to try to change how much society cares about animal welfare, and hope that TAI then reflects that. I guess this is the hope of many animal advocates. But I admit that seems too slow to work at this stage, so I agree that animal advocates should probably prioritize trying to influence those developing AI right now.