Hmm for some reason I feel like this will get me downvoted, but: I am worried that an AI with “improve animal welfare” built into its reward function is going to behave a lot less predictably with respect to human welfare. (This does not constitute a recommendation for how to resolve that tradeoff.)
I think this is exactly correct and I don’t think you should be downvoted?
Uh...this comment here is a quick attempt to try to answer this concern most directly.
Basically, longtermism and AI safety has the ultimate goal of improving the value of the far future, which includes all moral agents.
So in a true, deep sense, animal welfare must already be included. So instructions that sound like, “improve animal welfare”, should be accounted for already in “AI alignment”.
Now, despite the above most current visions/discussions of the far future that maximize welfare (“make the future good”) focuses on people. This focus on people seems reasonable for various reasons.
If you wanted to interrogate these reasons, and figure out what kind of people, what kind of entities, or what animals are involved, this seems to involve looking at versions of “Utopia”.
However, getting a strong vision of Utopia seems not super duper promising at the immediate moment.
The reason why it’s not promising is because of presentation reasons and the lower EV. Trying to have people sit around and sketch out Utopia is hard to do, and maybe we should just get everyone on board for AI safety.
This person went to a conference and wrote a giant paper (I’m not joking, it’s 72 pages long), to try to understand how to present this.
Because it is relevant (for example, to this very concern and many other issues in various ways) someone I know briefly tried to poke at work at “utopia” (they spent like a weekend on it).
To get a sense of this work, the modal task in this person’s “research” was a 1on1 discussion (with a person from outside EA but senior and OK with futurism). The discussions basically went like: ”Ok, exploring the vision of the future is good. But let’s never, ever use the word Utopia, that’s GG. Also, I have no idea how to start.”.
So this comment is answering: “What should we do about this issue about AI and animal welfare?”
Uh, basically, the thoughts in this comment here are necessarily meta.… apologies to your eyeballs.
Let’s treat this like field building
So to answer this question, it’s sort of good to treat this problem as early field building (even if it doesn’t shake down to a field or cause area).
It seems beneficial to have some knowledge of wild animal welfare, farmed animal welfare, and AI safety
And each major camp in them, e.g. the short timeline people, slow takeoff people, and the S-risk interests.
You should get a glance at the non-EA “AI ethics people” (their theory of change or worldview already appears in this post and comments).
Uh, it seems possible you might benefit from some domain knowledge of biology, animal welfare science, applied math and machine learning at various points.
So I’m saying, like, knowledge of some of the key literature, worldviews, subcultures.
Ideally, you would have a sense of how these orgs and people fit together and also how they might change or grow in the next few years, maybe.
Thoughts about why this context matters
So you might want to know the above because this would be a new field or new work and actual implementation matters. Issues like adverse selection, seating and path dependency is important.
Concrete (?) examples of considerations:
You want to figure out how viable and useful each instantiation of these current existing areas/beliefs and their people/institutions/interventions are, if you are trying to seat a new project or field in it.
These existing fields are overlapping and you could start in any number of them. For example S-risk matters a lot for “very short timelines”.
Where are the good people? What current leaders, cultures and do they have?
You can start in many different places and worldviews depending on the credence you have. You want to communicate with others well, even if they have different beliefs.
Like, seating the field firmly with one group of people (AI ethics) is not going to play super duper well with the EA or near-EA safety people.
Domain knowledge in fields like linguistics, machine learning and other knowledge is helpful to parse what is going on in these interventions:
For example, uh....I’m pretty uncertain or confused about the “communicate with animals” subthreads in this post:
I took a look at the whale one. So what’s going to be relevant is, like, linguistics and theory of mind, rather than “AI”.
The pivotal role of “machine learning” in some sort of unsupervised learning...is plausible I guess? But doesn’t seem likely to be the heart of the problem in communicating with whales. So that substantively knocks at the “AI element” here.
Most moral patients (IMO over >99.99%) aren’t going to be easy to communicate with, so I’m uncertain what the theory of change is here.
Maybe there’s instrumental value from nerdsniping? I think I’m OK nerdsniping people into animal welfare, but that’s complicated:
Some of the very top animal welfare leaders in EA that we look up to are basically Jedi Knights or Wallfacers, and that is a different pattern than you see from people who are really excited casually by ML or AI.
(I admit, it does seem sort of awesome to communicate with Octopuses, like how do you even feel bro, you’re awesome?)
I’m concerned with adverse selection and loss of focus
The animals that are being most grievously abused aren’t going to communicate with the same kind of expressiveness as whales.
I guess there’s still more. Like, divergences between animal welfare and other cause areas in some scenarios. I guess this is why I poked at here in this comment.
“So what? Why did read this comment? Give me some takeaways”
Probably talking to the S-risk people is a good start, like, message the people or the orgs in S-risk or sentience who commented here
Before jumping into using AI, I would get a little sense of the technical domains or theories of change involved, or speak to some experts, about the applications being proposed
I would try to stay concrete, which helps avoid the “very online” stuff
If you really had to get a project going or funded right away, I think audio and computer vision uses in factory farms is useful and has many applications (identifying suffering) and probably has the right mix of impact and shiny object.
It is a minor point But I would like to pushback on some misconceptions involving “panda conservation”, mostly by paraphrasing the relevant chapter from Lucy Cookes The Truth About Animals.
Contrary to headlines about libidoless pandas driving themselves extinct, the main reason pandas are going extinct is the main reason animal species in general are going extinct, habitat loss as humans take and fracture There land.
Giant Pandas almost entirely rely on bamboo for food , bamboo engages in synchronous flowering with the other bamboo plants in the area and then Seeds and dies off, because of this it is important that the pandas have a wide range of space they can travel across not only to mate with other pandas, but to access new bamboo forests when The ones they live in die.
These forests even in “ protected” areas, are threatened by mining, roads, and agriculture.
Meanwhile , giant pandas become an international symbol of China, China sends pandas to its allies as gifts, or loans them to foreign zoos at a million dollars per year( these rules also applying to any offspring born in the foreign countries), and panda cubs draw in domestic tourism, and large numbers are bred of an animal that doesn’t breed well in captivity, to release 10 socially maladjusted giant pandas 8 of which don’t survive.
Pandas aren’t hogging conservation dollars because
1)the money isn’t /conservation money/ that would go to other species, It’s politics and business
2)The benefits that would protect wild pandas, ( protecting large intact tracts of land) would also help a wide array of wildlife, this is a general trend of megafauna, they need more space and are disproportionately impacted by habitat loss, which is the leading cause of species extinction, and they are charismatic, functioning as umbrella species that protect whole ecosystems.
3) The most effective ways to save panda populations aren’t being acted upon in the first place
Side-note: I do think pandas are an obvious place to start when it comes to genetically modifying wildlife, considering they are a Charismatic Megafaunal Herbivore, that normally has twins , but always abandons one offspring in the wild( because bamboo is too low calorie compared to what it’s omnivore ancestors ate to feed both twins) modifying them to only produce one offspring at a time feels like a no-brainer assuming we can get There numbers up still.
My guess is that most money that is “raised using a picture of a Panda”, actually goes to conservation broadly.
Maybe advocacy that focuses on mega fauna is more mixed in value and not negative (but this seems really complicated and I don’t really have any good idea).
Finally, I didn’t read the article, but slurs against an animal species seems like really bad thinking. Claims that Pandas or other animals are to blame for their situation, are almost always a misunderstanding of evolution/fitness, because, as you point out, they basically evolved perfectly for their natural environment.
Hi Cate, thank you for your courage to express potentially controversial claims, and I upvoted (but not strongly) for this reason.
I am not a computer or AI scientist. But my guess is that you are probably right, if by “predictable” we mean “predictable to humans only”. For example, in a paper (not yet published) Peter Singer and I argue that self-driving cars should identify animals that might be on the way and dodge them. But we are aware that the costs of detection and computation will rise, and that the AI will have more constraints in its optimization problem. As a results the cars might be more expensive and they might be willing sacrifice some human welfare, such as by causing discomfort or scare to passengers while braking violently for a rat crossing.
But maybe this is not a reason to worry. If, like how most of the stakes/wellbeing lie in the future, most of the stakes and wellbeing lie with nonhuman animals, maybe that’s a bullet we need to bite. We (longtermists) probably wouldn’t say we worry that if an AI cares about the whole future it would be a lot less predictable with respect to the welfare of current people, we are likely to say this is how it should be.
Another reason to not over-worry is that human economics will probably constrain that from happening to a high extent. Using the self-driving car example again, if some companies’ cars care about animals, some don’t, the cars that don’t will, other things being equal, be cheaper and safer for humans. So unless we so miraculously convince all car producers to take care of animals, we probably won’t have the “problem” (which for me, that we won’t get “that problem” is the actual problem). The point probably goes beyond just economics, politics, culture, human psychology, possibly all have similar effects. My sense is that as far as humans are in control of the development of AI, AI is more likely to be too humancentric than not being humancentric enough.
Hmm for some reason I feel like this will get me downvoted, but: I am worried that an AI with “improve animal welfare” built into its reward function is going to behave a lot less predictably with respect to human welfare. (This does not constitute a recommendation for how to resolve that tradeoff.)
I think this is exactly correct and I don’t think you should be downvoted?
Uh...this comment here is a quick attempt to try to answer this concern most directly.
Basically, longtermism and AI safety has the ultimate goal of improving the value of the far future, which includes all moral agents.
So in a true, deep sense, animal welfare must already be included. So instructions that sound like, “improve animal welfare”, should be accounted for already in “AI alignment”.
Now, despite the above most current visions/discussions of the far future that maximize welfare (“make the future good”) focuses on people. This focus on people seems reasonable for various reasons.
If you wanted to interrogate these reasons, and figure out what kind of people, what kind of entities, or what animals are involved, this seems to involve looking at versions of “Utopia”.
However, getting a strong vision of Utopia seems not super duper promising at the immediate moment.
The reason why it’s not promising is because of presentation reasons and the lower EV. Trying to have people sit around and sketch out Utopia is hard to do, and maybe we should just get everyone on board for AI safety.
This person went to a conference and wrote a giant paper (I’m not joking, it’s 72 pages long), to try to understand how to present this.
Because it is relevant (for example, to this very concern and many other issues in various ways) someone I know briefly tried to poke at work at “utopia” (they spent like a weekend on it).
To get a sense of this work, the modal task in this person’s “research” was a 1on1 discussion (with a person from outside EA but senior and OK with futurism). The discussions basically went like:
”Ok, exploring the vision of the future is good. But let’s never, ever use the word Utopia, that’s GG. Also, I have no idea how to start.”.
Uh, so if you read the above (or just the 1st or 2nd layer deep of bullet points in the above comment), this raises questions.
Like, does this mean animal welfare boils down to AI safety?
What is the point of this post, really? What do we do?
So yeeeahhh....It’s pretty hard to begin.
Yeeeahhh.....
Like, so there’s a lot of considerations here.
So this comment is answering: “What should we do about this issue about AI and animal welfare?”
Uh, basically, the thoughts in this comment here are necessarily meta.… apologies to your eyeballs.
Let’s treat this like field building
So to answer this question, it’s sort of good to treat this problem as early field building (even if it doesn’t shake down to a field or cause area).
It seems beneficial to have some knowledge of wild animal welfare, farmed animal welfare, and AI safety
And each major camp in them, e.g. the short timeline people, slow takeoff people, and the S-risk interests.
You should get a glance at the non-EA “AI ethics people” (their theory of change or worldview already appears in this post and comments).
Uh, it seems possible you might benefit from some domain knowledge of biology, animal welfare science, applied math and machine learning at various points.
So I’m saying, like, knowledge of some of the key literature, worldviews, subcultures.
Ideally, you would have a sense of how these orgs and people fit together and also how they might change or grow in the next few years, maybe.
Thoughts about why this context matters
So you might want to know the above because this would be a new field or new work and actual implementation matters. Issues like adverse selection, seating and path dependency is important.
Concrete (?) examples of considerations:
You want to figure out how viable and useful each instantiation of these current existing areas/beliefs and their people/institutions/interventions are, if you are trying to seat a new project or field in it.
These existing fields are overlapping and you could start in any number of them. For example S-risk matters a lot for “very short timelines”.
Where are the good people? What current leaders, cultures and do they have?
You can start in many different places and worldviews depending on the credence you have. You want to communicate with others well, even if they have different beliefs.
Like, seating the field firmly with one group of people (AI ethics) is not going to play super duper well with the EA or near-EA safety people.
Domain knowledge in fields like linguistics, machine learning and other knowledge is helpful to parse what is going on in these interventions:
For example, uh....I’m pretty uncertain or confused about the “communicate with animals” subthreads in this post:
I took a look at the whale one. So what’s going to be relevant is, like, linguistics and theory of mind, rather than “AI”.
The pivotal role of “machine learning” in some sort of unsupervised learning...is plausible I guess? But doesn’t seem likely to be the heart of the problem in communicating with whales. So that substantively knocks at the “AI element” here.
Most moral patients (IMO over >99.99%) aren’t going to be easy to communicate with, so I’m uncertain what the theory of change is here.
Maybe there’s instrumental value from nerdsniping? I think I’m OK nerdsniping people into animal welfare, but that’s complicated:
Some of the very top animal welfare leaders in EA that we look up to are basically Jedi Knights or Wallfacers, and that is a different pattern than you see from people who are really excited casually by ML or AI.
(I admit, it does seem sort of awesome to communicate with Octopuses, like how do you even feel bro, you’re awesome?)
I’m concerned with adverse selection and loss of focus
Charismatic mega fauna are a focus already (literally too much, according to even some non-EAs), and it’s unclear how more attention on those animals will most usefully help animal welfare.
The animals that are being most grievously abused aren’t going to communicate with the same kind of expressiveness as whales.
I guess there’s still more. Like, divergences between animal welfare and other cause areas in some scenarios. I guess this is why I poked at here in this comment.
“So what? Why did read this comment? Give me some takeaways”
Probably talking to the S-risk people is a good start, like, message the people or the orgs in S-risk or sentience who commented here
Before jumping into using AI, I would get a little sense of the technical domains or theories of change involved, or speak to some experts, about the applications being proposed
I would try to stay concrete, which helps avoid the “very online” stuff
If you really had to get a project going or funded right away, I think audio and computer vision uses in factory farms is useful and has many applications (identifying suffering) and probably has the right mix of impact and shiny object.
It is a minor point But I would like to pushback on some misconceptions involving “panda conservation”, mostly by paraphrasing the relevant chapter from Lucy Cookes The Truth About Animals.
Contrary to headlines about libidoless pandas driving themselves extinct, the main reason pandas are going extinct is the main reason animal species in general are going extinct, habitat loss as humans take and fracture There land.
Giant Pandas almost entirely rely on bamboo for food , bamboo engages in synchronous flowering with the other bamboo plants in the area and then Seeds and dies off, because of this it is important that the pandas have a wide range of space they can travel across not only to mate with other pandas, but to access new bamboo forests when The ones they live in die.
These forests even in “ protected” areas, are threatened by mining, roads, and agriculture.
Meanwhile , giant pandas become an international symbol of China, China sends pandas to its allies as gifts, or loans them to foreign zoos at a million dollars per year( these rules also applying to any offspring born in the foreign countries), and panda cubs draw in domestic tourism, and large numbers are bred of an animal that doesn’t breed well in captivity, to release 10 socially maladjusted giant pandas 8 of which don’t survive.
Pandas aren’t hogging conservation dollars because 1)the money isn’t /conservation money/ that would go to other species, It’s politics and business 2)The benefits that would protect wild pandas, ( protecting large intact tracts of land) would also help a wide array of wildlife, this is a general trend of megafauna, they need more space and are disproportionately impacted by habitat loss, which is the leading cause of species extinction, and they are charismatic, functioning as umbrella species that protect whole ecosystems.
3) The most effective ways to save panda populations aren’t being acted upon in the first place
Side-note: I do think pandas are an obvious place to start when it comes to genetically modifying wildlife, considering they are a Charismatic Megafaunal Herbivore, that normally has twins , but always abandons one offspring in the wild( because bamboo is too low calorie compared to what it’s omnivore ancestors ate to feed both twins) modifying them to only produce one offspring at a time feels like a no-brainer assuming we can get There numbers up still.
Yes, everything you said sounds correct.
My guess is that most money that is “raised using a picture of a Panda”, actually goes to conservation broadly.
Maybe advocacy that focuses on mega fauna is more mixed in value and not negative (but this seems really complicated and I don’t really have any good idea).
Finally, I didn’t read the article, but slurs against an animal species seems like really bad thinking. Claims that Pandas or other animals are to blame for their situation, are almost always a misunderstanding of evolution/fitness, because, as you point out, they basically evolved perfectly for their natural environment.
Thanks for this excellent note.
Hi Cate, thank you for your courage to express potentially controversial claims, and I upvoted (but not strongly) for this reason.
I am not a computer or AI scientist. But my guess is that you are probably right, if by “predictable” we mean “predictable to humans only”. For example, in a paper (not yet published) Peter Singer and I argue that self-driving cars should identify animals that might be on the way and dodge them. But we are aware that the costs of detection and computation will rise, and that the AI will have more constraints in its optimization problem. As a results the cars might be more expensive and they might be willing sacrifice some human welfare, such as by causing discomfort or scare to passengers while braking violently for a rat crossing.
But maybe this is not a reason to worry. If, like how most of the stakes/wellbeing lie in the future, most of the stakes and wellbeing lie with nonhuman animals, maybe that’s a bullet we need to bite. We (longtermists) probably wouldn’t say we worry that if an AI cares about the whole future it would be a lot less predictable with respect to the welfare of current people, we are likely to say this is how it should be.
Another reason to not over-worry is that human economics will probably constrain that from happening to a high extent. Using the self-driving car example again, if some companies’ cars care about animals, some don’t, the cars that don’t will, other things being equal, be cheaper and safer for humans. So unless we so miraculously convince all car producers to take care of animals, we probably won’t have the “problem” (which for me, that we won’t get “that problem” is the actual problem). The point probably goes beyond just economics, politics, culture, human psychology, possibly all have similar effects. My sense is that as far as humans are in control of the development of AI, AI is more likely to be too humancentric than not being humancentric enough.