If you don’t mind, I’ll copy the two parts that stood out to me the most and helped to clarify the point for me better. If these points are valid, and I do think the logic makes sense, then this is quite concerning. Would love to hear other peoples thoughts on this.
And here’s the crux: If it (AGI) arrives, it may lock-in the values that exist at the time, including how we think about and treat animals on factory farms. This is because an AGI could be coded to reflect the preferences of the programmer—a potentially powerful individual or institution, since it’s unlikely this technology will emerge in a decentralized way given the capital and technical expertise required to build it—for the purpose of assisting them in achieving their and what they believe should be society’s goals, and one of those goals might be raising animals for food. What’s more, an AGI would be able to figure out how to farm animals in even more efficient ways, decreasing the cost of meat—which most people would celebrate—and increasing the profit margins of those who stand to benefit from this technology. No human would be more powerful than an AGI, so whatever force aims an AGI would have more power than any force that does not have that ability.
This value lock-in, combined with the fact that an AGI would not be hard to replicate, makes it such that the values encoded into the AGI could exist for as long as the universe can support life. As MacAskill writes, “There’s nothing different in principle between the software that encodes Pong and the software that encodes an AGI. Since that software can be copied with high fidelity, an AGI can survive changes in the hardware instantiating it. AGI agents are potentially immortal.”
I agree that this is an important issue and it feels like the time is ticking down on our window of opportunity to address it. I can imagine some scenarios in which this value lock in can play out.
At some point, AGI programmers will reach the point where they have the opportunity to train AGI to recognize suffering vs happiness as a strategy to optimize it to do the most good. Will those programmers think to include non-human species? I could see a scenario where programmers with human-centric world views would only think to include datasets with pictures and videos of human happiness and suffering. But if the programmers value animal sentience as well, then they could include datasets of different types of animals as well!
Ideally the AGI could identify some happiness/suffering markers that could apply to most nonhuman and human animals (vocalizations, changes in movement patterns, or changes in body temperature), but if they can’t then we may need to segment out different classes of animals for individual analysis. Like how would AGI reliably figure out when a fish is suffering?
And on top of all this, they would need to program the AGI to consider the animals based on moral weights, which we are woefully unclear on right now.
There is just so much we don’t know about how to quantify animal suffering and happiness which would be relevant in programming AGI. It would be great to be able to identify these factors so we can eventually get that research into the hands of the AGI programmers who become responsible for AI take-off. Of course, all this research could be for negligible impact if the key AGI programmers do not think animal welfare is an important enough issue to take on.
Are there any AI alignment researchers currently working on the issue of including animals in the development of AI safety and aligned goals?
Of course, all this research could be for negligible impact if the key AGI programmers do not think animal welfare is an important enough issue to take on.
Exactly what I was thinking too. Unfortunately I think AGI will (and likely already is) move at light speed compared to the inclusion of animal consideration in our moral circle (when has tech not greatly exceeded pace with social movements?). If there’s going to be a lock in, I’m fairly confident it’s going to be well before we’ll be where we need to be with our relationship with animals— even if we abolish factory farming by then.
So where does that leave us? Infiltrate companies working on AGI? Bring them into our circles and engage in conversations? Entice programmers/researchers with restricted grants (to help shape those datasets)? Physically mail them a copy of Animal Liberation? Are we even ready to engage in a meaningful way?
There’s just so many questions. Really thought-provoking stuff.
Are there any AI alignment researchers currently working on the issue of including animals in the development of AI safety and aligned goals?
Would love to know this too! I’m fairly new to this world and still poking around and learning, if I dig anything up I’ll edit this post.
Thanks for sharing Brian!
If you don’t mind, I’ll copy the two parts that stood out to me the most and helped to clarify the point for me better. If these points are valid, and I do think the logic makes sense, then this is quite concerning. Would love to hear other peoples thoughts on this.
I agree that this is an important issue and it feels like the time is ticking down on our window of opportunity to address it. I can imagine some scenarios in which this value lock in can play out.
At some point, AGI programmers will reach the point where they have the opportunity to train AGI to recognize suffering vs happiness as a strategy to optimize it to do the most good. Will those programmers think to include non-human species? I could see a scenario where programmers with human-centric world views would only think to include datasets with pictures and videos of human happiness and suffering. But if the programmers value animal sentience as well, then they could include datasets of different types of animals as well!
Ideally the AGI could identify some happiness/suffering markers that could apply to most nonhuman and human animals (vocalizations, changes in movement patterns, or changes in body temperature), but if they can’t then we may need to segment out different classes of animals for individual analysis. Like how would AGI reliably figure out when a fish is suffering?
And on top of all this, they would need to program the AGI to consider the animals based on moral weights, which we are woefully unclear on right now.
There is just so much we don’t know about how to quantify animal suffering and happiness which would be relevant in programming AGI. It would be great to be able to identify these factors so we can eventually get that research into the hands of the AGI programmers who become responsible for AI take-off. Of course, all this research could be for negligible impact if the key AGI programmers do not think animal welfare is an important enough issue to take on.
Are there any AI alignment researchers currently working on the issue of including animals in the development of AI safety and aligned goals?
Agree with the sentiment, thanks for the reply!
Exactly what I was thinking too. Unfortunately I think AGI will (and likely already is) move at light speed compared to the inclusion of animal consideration in our moral circle (when has tech not greatly exceeded pace with social movements?). If there’s going to be a lock in, I’m fairly confident it’s going to be well before we’ll be where we need to be with our relationship with animals— even if we abolish factory farming by then.
So where does that leave us? Infiltrate companies working on AGI? Bring them into our circles and engage in conversations? Entice programmers/researchers with restricted grants (to help shape those datasets)? Physically mail them a copy of Animal Liberation? Are we even ready to engage in a meaningful way?
There’s just so many questions. Really thought-provoking stuff.
Would love to know this too! I’m fairly new to this world and still poking around and learning, if I dig anything up I’ll edit this post.