www.jimbuhler.site
Also on LessWrong and Substack, with different essays.
Jim Buhler
Karma: 1,187
I assume the most important reason is that it is something that most people close to them do. Likewise, I think most people prioritise animals with a higher probability of sentience like chickens instead of shrimps because it is what most people close to them do.
Interesting. I think there’s something to this analogy, though ofc the social pressure to put your seatbelt on is far higher than that to prioritize chickens over shrimp.
I guess [their motivation] has little to do with the actual probability of sentience of the animals in question.
Yeah, maybe they just rationalize their motivations with moral weight arguments while their real drive is something else (see Simler & Hanson 2018). And highlighting potential biases we have might be helpful. On the other hand, you may wanna mainly stick to red-teaming the importance of p(sentience) as a potential crux (by, e.g., red-teaming Clatterbuck and Fischer), anyway, if that’s the reason people give (even if it might not be their real motivation deep down). I generally find this to be the most productive. People rarely update just based on noticing or being reminded of a bias they may have.
I guess most people see voting as fulfilling their duty to improve society.
That also seems part of the picture, yeah! And notice that this bolsters my broader point that it might not be about EV max and that there might be no inconsistency between voting and being difference-making risk-averse.
I wonder to what extent people donate to interventions targeting animals which are more likely to be sentient to boost the probability of increasing welfare. People routinely take actions which are super unlikely to actually matter
This position many animal advocates hold (even if only implicitly) was indeed rationalized/explained with difference-making risk aversion by Clatterbuck and Fischer (2025). And in this case, p(sentience), and moral weights more broadly, indeed seem important, actually.
I think it’s very plausible people are inconsistent in how difference-making risk averse they are for different things. However, let me play devil’s advocate:Seatbelts. One could argue this is just a habit they just don’t bother questioning, not risk-neutral EV max and getting mugged by small probabilities.
Voting. I would be surprised if many of the people who prioritize chickens because of risk aversion do vote. If they do, I agree this seems inconsistent. But, fwiw, if they were forced to pick a lane, I think most would drop voting and not their diffence-making risk aversion.
(Tangential but I guess from the above that you think the following is not another example where MNB is sensitive to the individuation of normative views, and I’d like to understand why. Nw at all if you don’t have the time to reply, tho.)
Antonia found an intervention that reduces overall animal suffering in the near-term, but she’s not sure which is true betweenL) the long-term effects dominate, but I don’t know what they overall imply, and I can’t ignore them (so I’m clueless).
N) neartermism thanks to bracketing out the long-term effects (so I should intervene).
Brian comes along and says he agrees with the above and subdivides L, this way:
L) the long-term effects dominate, but he doesn’t know what they overall imply, and he can’t ignore them (so he’s clueless).
L1) same, but he trusts his longtermist best guess that the intervention is bad, assuming pure negative utilitarianism (so he should not intervene).
L2) same but assuming negative-leaning utilitarianism (so he should not intervene).
L3) he trusts his longtermist best guess that the intervention is good, assuming classical utilitarianism (so he should intervene).
N) neartermism thanks to bracketing out the long-term effects (so he should intervene).
Antonia shares Brian’s above best guesses and normative uncertainty. They both totally agree. The only difference is that Brian specified normative sub-views.
Now, say Nuutti joins the party, agrees with these two, but recategorizes things this way:
L1) stubborn precise EV despite imprecision arguments + negative utilitarianism (we should not intervene)
O) all other plausible normative views (in sum, we’re clueless)
The MNB sceptic would say that Antonia grouping L1-3 together to form L is just as arbitrary as Nuutti grouping L2, L3, and N together to form O.[1]
Is your response: The former seems less arbitrary becauseL1-3 share key epistemic principles and/or decision theory that make L an actual normative view (even though the moral theory part is imprecise). In contrast, L2, L3, and N have nothing in common, normatively that justifies grouping them against L1. It’d be too arbitrary to consider N + L2 + L3 as a normative view.
Normative views seem to be the most legitimate units to bracket over (e.g., more legit than empirical views). Making a comprehensive case for/against this would be nice, but I give some reasons for, in this section.
- ^
With the consequentist-bracketing version of the individuation problem I present here, the bracketer can appeal to a “only value locations that have been identified can be bracketed in” principle. This saves them if this principle is sound. Here, this doesn’t save them. The normative theories have been identified in both cases.
The idea that the unpleasantness of pain increases superlinearly with its intensity (i.e. an 8⁄10 on the pain scale is more than twice as bad as a 4⁄10).
Yeah… I wish we would just say that the 4 is actually lower than 4 and directly track what you mean by “unpleasantness” with these scores, since this is what we care about. But that’s not how people use the /10 scale, unfortunately. And that’s understandable. If they were, they would seldom say that they’re suffering above a 1⁄10.[1]
And yes. When researchers/people assign welfare ranges, they think they’re tracking “unpleasantness”, but I also suspect they are actually tracking what you mean by “intensity” to a large extent, which may lead to very misguided cross-species welfare tradeoffs. I am extremely skeptical of the following counter-view you describe:If a researcher judges an animal to be at 10% of its capacity, they simply mean 1⁄10 as bad as its worst state — there’s no question about whether 100% is “really” 10x worse, because that’s just what the numbers mean by construction.
Maybe that’s what they mean, but I doubt that their estimate is not deeply biased by the “unpleasantness”/”intensity” confusion.
To be clear, though, I don’t want people to take away that we should care less about insects and shrimp. There are so many other considerations. If anything, this should make us less confident in precise-ish moral weight estimates (and maybe look for projects robust to this uncertainty).That’s a very important problem you raise! Thank you for this. :)
Great points from you here and from @Mia Fernyhough in another thread! What about in countries where animal advocacy is (almost) nonexistent and where the counterfactual is probably not cage-free, but no change at all? Curious what the two of you (and others) think. I know this does not address all the limitations you raise, but maybe the most crucial ones?
The chatbot completely misses DiGiovanni’s point fwiw aha. Literally all of the objections it raises are explicitly addressed in what I’ve linked or elsewhere in his sequence. :)
No pressure to read anything, though. It’s a thorny topic and understanding all the complex details takes time.
This [post-keynesian economics] literature has produced not just a diagnosis of the [cluelessness] problem but a set of practical heuristics and institutional responses that could meaningfully supplement EA analysis in situations of deep uncertainty.
Fwiw, DiGiovanni and myself argue that following such heuristics is not an appropriate response to the deep-uncertainty situation EAs (at least impartial ones) face. We don’t directly respond to the literature you cite, but rather to the arguments found in the following refs you might be interested in: Thorstad & Mogensen 2020; Tomasik 2015; The Global Priorities Institute 2024, §§1.2.1 and 4.2.1; Grant & Quiggin 2013.
Thanks for posting this :)
Fwiw, one can very well agree that all pains are comparable in theory, but that the difference between a pinprick and genuine torture is so large that, in practice, the latter will often dominate. I find this harder to “debunk” than antiaggregationism.
Given our deep uncertainty on i) how many pinpricks outweigh torture and ii) moral weights and welfare ranges,[1]I certainly don’t find it implausible that nematodes, shrimp, or even chickens have experiences that are too mild, relative to other beings, to dominate EV calculations—despite their high numbers and assuming aggregationism.[2]
So sure, maybe, in principle, there is a number of warmed up nematodes that outbalances 1 trillion human-years of extreme torture. But this says nothing about tradeoffs we can(not) make between humans and nematodes in the real world.- ^
Well, (i) matters only insofar as it is relevant to (ii), here, but I thought I’d acknowledge (i) separately, still.
- ^
And you said things that suggest you agree in this recent interview. You seemed to have deviated from your previous “nematodes (almost) surely dominate” view. Or did I miss something?
- ^
I recommend decreasing the uncertainty about how the individual (expected hedonistic) welfare per unit time of different organisms and digital systems compares with that of humans. In particular, I recommend supporting Rethink Priorities (RP) via restricted funding. [...] I committed donating 2 k$ to RP for them to scope out whatever projects they believe would decrease the most cost-effectively the uncertainty about how the individual welfare per unit time of different organisms and digital systems compares with that of humans.
Do you think “whatever projects RP believes would decrease the most cost-effectively the uncertainty” would address your reasons for uncertainty? I haven’t yet taken the time to comprehend the details of your current views on hedonistic P(sentience)-adjusted welfare ranges, but I get the sense that your sources of uncertainty on this are not the same as RP’s.
Thanks Wladimir! And do you think Birch et al. meant something different from what you describe with the term acuity (instead of resolution)?
Evaluative richness can be further subdivided into evaluative bandwidth and evaluative acuity: “Rich affect-based decision making takes many inputs into account at once (evaluative bandwidth) and is sensitive to small differences in those inputs (evaluative acuity)” ([Birch, Schnell, & Clayton 2020]).
(Emphases are mine.)
Is the bandwidth-acuity distinction the same as the range-resolution one in Alonso & Schuck-Paim (2025)?Range: The maximum ‘absolute’ value of an affective experience that an organism can perceive, defining the absolute amplitude between the highest or lowest affective intensities possible. A high-range system allows for extreme Pain or pleasure, which may be necessary for strong motivational reinforcement, while a low-range system limits organisms to milder affective states.
Resolution: How precisely an organism can differentiate between varying levels of affective intensity. A high-resolution system allows for fine distinctions, while a low-resolution system may encode only broad categories, limiting the precision of affective experiences
To explore whether high-intensity Pain can be experienced by primitive sentient organisms, we reframe the question as one of information resolution within a scale of Pain intensities. Briefly, two evolutionary possibilities are considered for how nervous systems evolved to represent varying Pain intensities, enabling appropriate responses to competing behavioural demands: (1) increasing resolution within a fixed range of intensities (i.e., introducing finer gradations between the minimum and maximum perceived intensity, e.g., 0 to 10) or (2) expanding the range itself of perceived intensities (e.g., extending the maximum perceived intensity value beyond 10, to values like 100, or 1000)
Were you aware of the following? (From Shriver 2024, §3.2.6.)
it’s not immediately obvious that simply having a more discerning ability to evaluate will also entail having the capacity for more intense experiences. Henry Shevlin, in an unpublished draft, proposed two possible accounts of what happens when we add evaluative richness. On the “compression account,” the ends of the spectrum stay as far apart as they were previously, but we add precision in how small the units are that exist between the two poles. However, on the “expansion account,” adding more evaluative states pushes the ends of the spectrum outward. As such, on the expansion account, adding richness would seem to actually increase the range of evaluative states and hence would give us one account of how adding richness could increase a welfare range. To be clear, there was no particular reason to assume that the expansion rather than the compression account is correct, but this does offer one story of how increased neuron counts in affective parts of the brain may increase welfare capacity.
In the above quotes, Shevlin and you are basically saying the same things with different terms, right? Or am I missing something?
(Great post. Thanks for sharing!)
On the cost point — Right, the words I chose made it very unclear whether and when I was talking about only costs, or only benefits, or overall fitness once we combine both, sorry.
On my contradiction — Oops yeah, I meant organisms with lower resolution. My bad.
Thanks for taking the time to reply to all this. Very helpful!
(Sorry for commenting only 3 years after you posted.) ^^
1.8kg of chicken would be replaced by 108g of shrimp which is roughly equal to 7 farmed shrimp
So, in your model, the upside of helping chickens dominates the downside of increasing shrimp consumption at least as long as chickens matter ~seven times more than shrimp.
I am overall highly certain that the change will have the expected effects of improving animal welfare significantly and that substitution effects will not cause this intervention to cause a net reduction in animal suffering.
So you were highly certain that chickens matter at least seven times more than shrimp? Are you, still? Your argument was:
The [mean P(Sentience)-adjusted] welfare range of shrimp is 0.031, whereas for chicken it is 0.332 (Fischer, 2023), suggesting that shrimp have 10% the capacity of pain of [chicken].
So then chickens would matter ten times more than shrimp. But how much weight would you put on the above argument? I could now make the opposite argument based on Rethink Priorities’ latest ranges where mean for shrimp is 0.08 and not 0.031), which would make chicken matter only four times more than shrimp (such that the intervention seems net negative according to your model).
Oh yes, I agree with all that. Just to make sure I understand what you think of my original point:
If affective states are whole-organism control states rather than simple sensory readouts, then escalating intensity plausibly requires extra integration, valuation, or modulatory capacity. In that case, intensity beyond “loud enough” would not be strictly neutral, and drift would be limited.
To be clear, extra integration, valuation, and modulatory capacity are costly only if they decrease fitness in some way, right? An unnecessarily louder alarm for some problem hurts only if it impedes your capacity to solve other important problems.
My original suggestion was that, while an unnecessarily loud alarm would generally be maladaptive because of the above (as your post suggests),[1] it might not in genuinely catastrophic situations, specifically. Because the importance of solving the problem signaled by the alarm overwhelmingly dominates. It matters little whether it impedes your capacity to solve other problems.
I get the impression that you agree with this as presently stated (at least depending on one’s interpretation of “overwhelmingly” and “little”) and were simply making sure I wasn’t taking away too much from my point. Is that correct? Or do you in fact see reasons to disagree with the above?[2]So I see neutral drift as a live alternative, but not the default. The framework is meant to clarify when neutrality is plausible versus when selection should instead cap, reshape, or avoid extreme intensity altogether.
Yes, absolutely. I did not mean to question any of that. I’m just curious about this potential specific deviation from the default in high-stakes situations.[3]
- ^
So, yes, I’m not claiming that “extreme felt intensity actually [often] falls into that neutral regime”, in that sense.
- ^
One counter-argument I see is that, while the unecessarily loud alarm doesn’t hurt in genuinely catastrophic situations, there always are risks of false alarm. If the loud alarm misfires, there is no overwhelmingly important problem that largely dominates the cost of reducing your capacity to solve others. This is just purely bad for fitness. I don’t know how significant this misfiring risk is, though, and hence how much weight to put on this counter-argument.
- ^
I’m curious because it seems highly relevant to, e.g., the question of whether organisms with
narrower welfare ranges(EDIT: lower resolution) could feel extreme pain, and how reliably we can estimate the probability of this, which in turn matters for how precise our moral weight estimates can be (see #3 in this informal research agenda).
- ^
I agree that in genuinely catastrophic situations, evolution should tolerate very “loud” alarms. The open question, though, is whether those alarms need to be implemented as extreme affective states, rather than through non-affective or lower-intensity control mechanisms.
I was assuming they do not need to be, but might appear and remain anyway if they have no significant downside, like, e.g., humans’ protruding chins. How loud the alarm is beyond the “loud enough” point would then just be a matter of luck.[1] Both just-loud-enough alarms and unnecessarily loud ones would be about equally effective so equally likely, all else equal. How plausible do you think this is?
Sorry for not being clear in my first comment, and thanks for helping pin down the crux!
Brilliant work — thanks for sharing.
On the costs of high-intensity affective states, which you suggest are high (such that we’d need a special explanation for why they exist):In affective neuroscience, core emotions are understood as whole-organism control states that recruit neuroendocrine, autonomic, and motivational systems, reorganizing behavior and physiology in ways that may be adaptive in the short term but biologically consequential if prolonged (Panksepp, 1998; McEwen, 2007). Prolonged or poorly regulated aversive states can interfere with feeding, reproduction, immune function, and behavioral flexibility, and may increase vulnerability to maladaptive stress responses. Empirical studies across taxa show that sustained aversive states are associated with measurable physiological and fitness trade-offs, including reduced growth and altered reproductive behavior (Sneddon et al., 2014). While such findings do not establish the evolutionary costs of extreme pain directly, they support the broader conclusion that escalating affective intensity is unlikely to be biologically neutral. From this perspective, high-intensity pain is not a cost-free refinement of sentience. Even if the neural machinery required to generate affect is relatively inexpensive, maintaining access to extreme affective intensities possibly represents an additional evolutionary investment whose persistence depends on whether, in a given life history and ecological context, its marginal adaptive benefits plausibly outweigh its marginal biological and modulatory costs.
In the highest-stakes situations from the point of view of evolution (possibly imminent death or failure to procreate), aren’t these costs of extreme states too negligible to make any non-trivial difference? The more the replication of the species is on the line, the more it’s fine if the alarm is unnecessarily loud.
Even if we grant that punishment is more effective than positive reward in shaping behavior, what about the consideration that once the animal learns, it’ll avoid situations where it gets punished, but it actively seeks out (and gets better at) obtaining positive reward?
Fair point, though then:
Any being that is motivated by severe pain where yours is motivated by pleasure (or lighter pain like small frustration, indeed) should be selected for over yours.
Your animal will presumably still need reminders of what it feels like not to avoid these situations to actually be motivated to avoid them. (Unless the suffering it felt the last time was so traumatizing that it’ll never make the mistake again but then, this hardly goes against the suffering-prevalence thesis.)
We know (from empirical findings, this time) that many of those pain-inducing situations are common and hard to (systematically) avoid.
(EDIT to add:) Why would your learning animal need rewards if it can just not repeat past mistakes? Maybe learning abilities say things about how large the welfare range is more than about pain vs. pleasure (see Schukraft et al. 2024, sec. 3.1.4, and 4.4.3).
But overall I’d be pretty hesitant to give much weight to theoretical arguments of this sort, especially since you can sometimes think of counterconsiderations like the one above.
In absolute terms, fair. I’m just skeptical that judgment calls on net welfare after empirically studying the lives of wild animals are any better. If there’s a logical or evolutionary reason to expect X, this seems like a stronger reason for X than “we’ve looked at what some wild animals commonly experience and we feel like what we see means X.”
Maybe stronger does not mean strong in absolute, though. But then, the conclusion would not be that we shouldn’t update much based on theoretical arguments of this sort, but that there is no evidence we can find (whether theoretical or empirical) on which we could base significant updates.And as a possible counterpoint to the premise, I remember this review of a book on parenting and animal training where it says that training animals with attention on positive reward (but also trying not to reward undesired behavior) works best. That’s a different context than evolution’s, though.
Interesting, I’ll look into this. Thanks!
I think we’d get the best sense of net wild animal welfare not from abstract arguments but by studying individual animals up close.
Tangential to your main point, but I’m actually not sure about this. For example, O’Brien (2021, §2.4) makes a theoretical argument according to which suffering is indirectly selected for by evolution. In this paper draft (sec. 5), I make a similar argument, defending that suffering is directly selected for (although it is partly based on the empirical findings that punishment is often more effective than reward at motivating behaviors). I think this kind of “suffering is a feature, not a bug” argument might in fact be more robust than our impression after looking at empirical stuff,[1] although I’d need to think more about this.
Interesting post, Lukas! Thanks for writing this :)- ^
Related thoughs (on the potential strengh of logical and evolutionary arguments, in particular) in the second paragraph of this section.
- ^
I’m worried everyone will just agree that this seems unlikely. That’s a very high bar.
I think we don’t care about whether it “values animal welfare”. We care about what happens to animals. There are many very plausible worlds where these two are uncorrelated (just like in ours, where people have never valued AW that high and it has never been that bad for farmed animals, especially the smaller ones).
That’s my favorite version, but I’m worried it invites everyone to just agree on “we should have some extra animal-focused work, anyway” and not red-team each other deeply enough.
So here’s a minimal version I propose: AI safety work that helps humans also helps other animals, to some extent.
(The “to some extent” is optional. I added it to invite people to think about whether AIS helps other animals at all, and not just all agree over the uncontroversial and boring claim that “AIS helps humans more than animals”.)
I like this minimal formulation because
impossible to misinterpret.
it makes clear that the more we lean yes on this, the more AI safety work that helps humans is overall robustly positive, all else equal (e.g., it’d be robust to uncertainty on moral weights, on the expected size of different populations in the future, or on the sign of x-risk reduction). And I think the answer to this question is a crux for some people for (not) supporting AIS work. I feel like none of the versions you propose (or the original version) quite captures this as much as I’d like.
Thanks for asking us, Toby! Looking forward to this debate week :)