Yeah that seems correct and I think that my post tomorrow is probably guilty of this. I’d even say I’m doing it intentionally to try to pull the best new founders into this area. I think the actual outlook for helping animals is just terribly bleak across the board and that’s probably not going to inspire people to join welfare tech incubators.
Aidan Kankyoku
Karma: 441
lol I actually held it back a week because of some infohazard concerns and I’m kicking myself for not beating you to the punch
Bubble in the sense of railroad bubbles successfully building railroads we still use, dot-com bubble laying fiber optic cable, etc. It could be a perfectly wise action for us to direct most of our spare energy/resources in this direction for a while.
The early research behind leafleting was low-quality in a way that should have been noticeable, and a better-calibrated version of the movement would not have spent so much time and money on it. Whereas the cage-free bubble was a necessary way of getting a massive worldwide campaign off the ground. I’d like use to have higher standards for our fads than we did for leafletting but I think the fad phenomenon is inevitable for a movement as constrained on talent and resources as we are.
I think the bubble framing is a useful and accurate description of welfare tech excitement. I just don’t think that’s a bad thing. From a longer post on this topic I’m publishing tomorrow:
From the outside, it might seem like the EA wing of the animal advocacy movement goes through fads. That’s because it does. In each cycle, funders get excited about a shiny new object, and put out a call for established organizations and new startup founders to rush into that space by making it clear funding will be available for anyone who does. Vegan leafleting was one such fad, followed by corporate cage-free campaigns, then cultivated meat and alternative proteins in general. Politics is the current fad, and welfare innovation will be the next one.
This can be frustrating for people working on an intervention that never gets popular. But that doesn’t mean it’s actually suboptimal. Some of our fads have worked out, and others haven’t; so it goes.
Animal advocates have limited resources, both money and people, spread thin across researchers, campaigners, and grantmakers. It’s probably the case that we needed to put all of our spare energy into cage-free campaigning to get the ball rolling there, and indeed, while the cage-free fight has been a big success, it’s hardly over despite all the energy invested. I’m glad that increasing our political capacity has been a major influence for the last few years and that it will continue to get attention.
But I’m also glad that there’s about to be a major infusion of money and attention into welfare innovation. And because of that, it’s quickly becoming clear that the pace of this innovation will be limited not by funding, but by talent.
Stated Values, Revealed Habits: The Challenge of Measuring AI Preferences
You’re splitting this the same way I meant to, though the quote you point out wasn’t as clear as it could have been: in strictly verifiable domains, AIs will eventually be so much better than humans that having humans involved will only lead to worse decisions. But the line between verifiable and unverifiable is gradual and fuzzy. My expectation in practice is that humans will develop such a strong habit of deferring to AIs that even if there are still humans “in charge” of certain things in the sense that they could implement one decision or another, they will not effectively differentiate the domains in which they by their own lights should maintain informed control, and will end up giving over complete control.
Keeping humans in the loop is a bigger question! I treated “humans eventually wind up not in the loop” as a premise for this post in part because I think it’s the default and in part because my primary audience of people focused on anima welfare probably don’t have the means to help keep humans in control, but maybe that’s giving up too easily.
What would an animal-aligned AI be aligned to?
I have observed the same sharp distinction as @ElliotTep: EAs not focused on animal welfare accept that TAI will end factory farming with something like Matt Reardon’s 99% certainty, while most focused on animal welfare (myself included) feel the risk of factory farming being perpetuated into the future is unacceptably high. This is probably a mix of several factors:
Animal welfare people having some bias to think our work is still important
Animal welfare people having a lower threshold for unacceptably high risk of this particular outcome
EAs outside animal welfare having less firsthand experience of frustration at the ways humans today seem to favor meat from tortured animals over cultivated meat
Animal welfare people having thought less about what extremely weird posthuman futures look like and how strongly TAI might pull in that direction
EAs outside animal welfare being overly confident about the extremely weird posthuman future
I’m focused on animal welfare and have been for 12 years. But I’ve also read my Greg Egan and think it is less likely than not that biological humans will continue to exist into the far future. But I still believe that ensuring AI ends factory farming rather than perpetuating it should be a top focus area for EA. I might try to think through this and write something better, but off the cuff reasons:
It’s possible biological substrates are necessary for consciousness/sentience. My credence on this is maybe 15% which I consider very low given our cluelessness about consciousness. If silicon platforms can’t support consciousness, then Von Neumann probes will probably carry gene sequences to other planets to seed biological civilizations. Low probabilities of galaxy-scale factory farming still look very important!
Maybe the weird posthuman future will get the rest of the lightcone and leave earth for biologicals. In that case it could be a thousand years before digital minds colonize the nearest star system. That’s a lot of time where factory farming could still represent a share of conscious experience within an order of magnitude of today’s. Any model that discounts future beings even just due to uncertainty about our ability to affect their welfare seems likely to make factory farming remain a top priority.
Adding to what Other Aidan said, I think it’s a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from “very slightly helpful” to “very slightly harmful” as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is −10 units, the difference between A and B matters much more than between B and C.
Congrats/good luck on soon entering the workforce!
I think OP is a valuable contribution to an ongoing discussion, but I’d encourage you not to update too aggressively on any one post. There are good reasons that some of the most rigorous organizations in the space (CG) have made cage-free a top priority and ex ante you should think it unlikely that they all converged without good reasons– see e.g. @Vasco Grilo🔸’s comment above.
I strongly agree that the animal welfare implications of AI should be owned at least as much by the AI safety space as by the animal welfare space, not that there needs to be a hard distinction between the two but there is obviously some declarative truth to it. Animal suffering is among the greatest lock-in risks.
I’m worried that many people outside the AW space believe the end of factory farming is a foregone conclusion. At one forum in SF in February mixing leaders from AI safety and AW, many AIS folks came away at least partly convinced by the AW folks that this cannot be taken for granted. AIS needs to take seriously the inside view of AW leaders that AI will not necessarily solve FAW by default, not to mention WAW.
Fair, the semantics of “going well” ultimately don’t matter so I think this is clear.
Most animals are wild animals, so the answer to this question should focus on them.
I can imagine a future where most animals are farmed animals. I’m not saying it’s particularly likely, but if humans spread to other planets, I think we’re more likely to take factory farming with us than take nature with us. Farmed animals should be part of this convo.
From @Tristan Katz :
Does WAW dwarf FAW in expectation? Or is FAW still important to consider in this discussion?
I can imagine a future where most animals are farmed animals. I’m not saying it’s particularly likely, but if humans spread to other planets, I think we’re more likely to take factory farming with us than take nature with us. Farmed animals should be part of this convo imo.
How am I supposed to predict whether the solution will be good for animals if I have no idea what that solution will look like?
That about sums it up haha.
I find this compelling, if discouraging.
I’m curious about this choice:
I will assume that “going well” for animals means something like “the ratio of positive experiences to negative experiences among animals is both meaningfully higher than it is today, and is above 1 (i.e. net positive)”.
I agree that animal experience is below 1 right now. I don’t actually think that a continuation of the status quo is possible; for reasons explained by Ian Morris, I think we either see massive growth or some form of collapse.
That aside, though, crossing 1 seems like a slightly unfair threshold. If the current ratio is massively negative, would you stand by reducing it to something close to but below 1 not being “AGI went well for animals”?
This post is persuasive overall and caused me to update negatively on the debate motion.
I don’t think this is load-bearing but this sentence:
However, it is not the number of animals that matters here, but the scale of humans’ impact on them.
I’m not certain about this distinction. Humans increasing or decreasing the number might be the largest impact, and at the same time, this seems to me to imply a greater concern for anthropogenic harm than non-anthropogenic harm. Is that what you meant?
If you end up reading it I’d particularly value your feedback on how I’m striking this balance.