I graduated from Georgetown University in December, 2021 with degrees in economics, mathematics and a philosophy minor. There, I founded and helped to lead Georgetown Effective Altruism. Over the last few years recent years, I’ve interned at the Department of the Interior, the Federal Deposit Insurance Corporation, and Nonlinear.
Blog: aaronbergman.net
I do not accept premise 2:
For some small amount of intense suffering, there is always some sufficiently large amount of moderate suffering such that the intense suffering is preferable.
To be clear, I think this premise is one way of distilling and clarifying the (or ‘a’) crux of my argument and if I wind up convinced that the whole argument is wrong, it will probably be because I am convinced of premise 2 or something very similar
Wow, this is super exciting and thanks so much to the judges! ☺️
An interesting dynamic around this competition was that the promise of the extremely cracked + influential judging team reading (and implicitly seriously considering) my essay was a much stronger incentive for me to write/improve it than the money (which is very nice don’t get me wrong).[1]
I’m not sure what the implications of this are, if any, but it feels useful to note this explicitly as a type of incentive that could be used to elicit writing/research in the future
Insofar as I’m not totally deluding myself, I mean the altruistic impact of some chance of shaping the judges’ views as opposed to the possibility of seeming mildly clever to some high status figures
Interesting, thanks! I might actually sign up for the Arctic Archive thing! I don’t see you mention m-discs like this—any reason for that?
Also, do you have any takes on how many physical locations a typical X is stored in, for various X?
X could be:
A wikipedia page
An EA Forum post
A YouTube video
A book that’s sold 100/1k/10k/100k/1M copies
Etc
After thinking about this post (“Utilitarians Should Accept that Some Suffering Cannot be “Offset””) some more, there’s an additional, weaker claim I want to emphasize, which is: You should be very skeptical that it’s morally good to bring about worlds you wouldn’t personally want to experience all of
We can imagine a society of committed utilitarians all working to bring about a very large universe full of lots of happiness and, in an absolute sense, lots of extreme suffering. The catch is that these very utilitarians are the ones that are going to be experiencing this grand future—not future people or digital people or whomever.
Personally, they all desperately want to opt out of this—nobody wants to actually go through the extreme suffering regardless of what benefits await, and yet all work day in and day out to bring about this future anyway, condemning themselves to heaven and to hell.
Of course it’s not impossible that their stated morals are right and preferences are morally wrong, but my claim is that we should be very skeptical that the “skin in the game” answer is the wrong one.
And of course here I’m just supposing/assuming that these utilitarians have the preference to avoid this massive future altogether
The point, again, is that you should probably not work to build a future that you yourself would not want to experience just because some abstract arguments say that this is morally good This is a strong hint (not definitive, but strong) that those abstract arguments are wrong.
I disagree but think I know what you’re getting at and am sympathetic. I made the following to try to illustrate and might add it in to the post if it seems clarifying
I made it on a whim just now without thinking too hard so don’t necessarily consider the graphical representation on as solid footing as the stuff in the post
Thanks!
(Ideal): annihilation is ideally desirable in the sense that it’s better (in expectation) than any other remotely realistic alternative, including <detail broadly utopian vision here>. (After all, continued existence always has some chance of resulting in some uncompensable suffering at some point.)
Yeah I mean on the first one, I acknowledge that this seems pretty counterintuitive to me but again just don’t think it is overwhelming evidence against the truth of the view.
Perhaps a reframing is “would this still seem like a ~reductio conditional on a long reflection type scenario that results in literally everyone agreeing that it’s desirable/good?”
And I don’t mean this in the sense of just “assume that the conclusion is ground truth”—I mean it in the sense of “does this look as bad when it doesn’t involve anyone doing anything involuntary?” to try to tease apart whether intuitions around annihilation per se are to any extent “just” a proxy for guarding against the use of force/coercion/lack of consent.
Another way to flip the ‘force’ issue would be “suppose a society concludes unanimously including via some extremely deliberative process (that predicts and includes the preferences of potential future people) that annihilation is good and desired. Should some outside observer forcibly prevent them taking action to this end (assume that the observer is interested purely in ethics and doesn’t care about their own existence or have valenced experience)?”
I’ll note that I can easily dream up scenarios where we should force people, even a whole society, to do something against its will. I know some will disagree, but I think we should (at least in principle, implementation is messy) forcibly prevent people from totally voluntarily being tortured (assume away masochism—like suppose the person just has a preference for suffering that results in pure suffering with no ‘secretly liking it’ along for the ride)
(Uncompensable Monster): one being suffering uncompensable suffering at any point in history suffices to render the entire universe net-negative or undesirable on net, no matter what else happens to anyone else. We must all (when judging from an impartial point of view) regret the totality of existence.
These strike me as extremely incredible claims, and I don’t think that most of the proposed “moderating factors” do much to soften the blow.
This one I more eagerly bite the bullet on, it just straightforwardly seems true to me that this is possible in principle (i.e., such a world could/would be genuinely very bad). And relevantly, orthodox utilitarianism also endorses this in principle, some of the time (i.e. just add up the utils, in principle one suffering monster can have enough negative utility)!
Moral uncertainty might help if it resulted in the verdict that you all things considered should prefer positive-utilitarian futures (no matter their uncompensable suffering) over annihilation. But I’m not quite sure how moral uncertainty could deliver that verdict if you really regard the suffering as uncompensable. How could a lower degree of credence in ordinary positive goods rationally outweigh a higher degree of credence in uncompensable bads? It seems like you’d instead need to give enough credence to something even worse: e.g. violating an extreme deontic constraint against annihilation. But that’s very hard to credit, given the above-quoted case where annihilation is “obviously preferable”.)
I don’t have an answer to this (yet) because my sense is just that figuring out how to make overall assessments of probability distributions on various moral views is just extremely hard in general and not “solved”.
This actually reminds me of a shortform post I wrote a while back. Let me just drop a screenshot to make my life a bit easier in terms of formatting nested quotes:
I think this^ brief discussion of how the two sides might look at the same issue gets at the fundamental problem/non-obviousness of the matter pretty well.
I think a more promising form of suffering-focused ethics would explore some form of “variable value” approach, which avoids annihilationism in principle by allowing harms to be compensated (by sufficient benefits) when the alternative is no population at all, but introduces variable thresholds for various harms being specifically uncompensable by extra benefits beyond those basic thresholds. I’m not sure whether a view of this structure could be made to work, but it seems more worth exploring than pro-annihilationist principles.
I think we may just have very different background stances on like how to do ethics. I think that we should more strongly decouple the project of abstract object level truth seeking from the project of figuring out a code of norms/rules/de facto ethics that satisfies all our many constraints and preferences today. The thing you propose seems promising to me as like a coalitional bargaining proposal for guiding action in the near future, but not especially promising as a candidate for abstract moral truth.
Thanks, yeah I may have gotten slightly confused when writing.
Wikipedia screenshot:
Let P be the thing I said in the post:
If A ≻ B ≻ C, there’s some probability p ∈ (0, 1) where a guaranteed state of the world B is ex ante morally equivalent to “lottery p·A + (1-p)·C”
or, symbolically
and let
I think but not in general.
So my writing was sloppy. Super good catch (not caught by any of the various LLMs iirc!)
But for the purposes of the argument everything holds together because you need independence axiom for VNM to hold. But still, sloppy.
Me: “This arbitrariness diminishes somewhat (though, again, not entirely) when viewed through the asymptotic structure. Once we accept that compensation requirements grow without bound as suffering intensifies, some threshold becomes inevitable. The asymptote must diverge somewhere; debates about exactly where are secondary to recognizing the underlying pattern.”
You:
“Grow without bound” just means that for any M, we have f(X) > M for sufficiently large X. This is different from there being a vertical asymptote so a threshold is not inevitable. For instance one could have f(X) = X or f(X) = X^2.
Straightforward error by me, I will change the wording. Not sure how that happened
It would be confusing to call this behavior continuous, because (a) the VNM axiom you reject is called continuity and (b) we are not using any other properties of the extended reals, but we are using real-valued probabilities and x values.
Yeah, idk, English and math only provide so many words. I could have spent more words more driving home and clarifying this point or inventing and defining additional terms. My intuition is that it’s clear enough as is (evidently we disagree about this) but if a couple other people say “yeah this is misleading and confusing” then I’ll concede that I made a bad choice about clarity vs brevity as a writing decision.
Ngl I am pretty confused about everything starting here. I think I’m just reading you wrong somehow. Like the difference in those magnitudes is huge, point taken, but I don’t see why that matters for my argument.
Moving from 10^10^10 to infinity, we would then believe that suffering has a threshold t where t + epsilon intensity suffering cannot be offset by removing t—epsilon intensity suffering
Confused here because yeah clearly adding t+epsilon and removing t-epsilon gives you a net change below zero. But I sense you might be getting at the (very substantive and important) cluster of critiques I respond to in this comment (?)
also need to propose some other mechanism like lexicographic order for how to deal with suffering above the infinite badness threshold.
Yeah I’m ~totally agnostic about this in the post. There are many substantively different possibilities about what the moral world might be like when dealing above that threshold, I agree! Could be distinct levels of lexicality, perhaps some literal integer like 13 levels or perhaps arbitrarily many. Probably other solutions/models as well
Maybe I should just remove/modify the “write down”, “compute”, and “physically instantiate” bit of rhetorical flourish because it might be doing more harm than good.
(Note that it may take me some time to update the post to reflect sections 1 and 2 in this comment)
Again, sharp eye, thanks for the comment!
I am getting really excellent + thoughtful comments on this (not just saying that) - I will mention here that I have highly variable and overall reduced capacity for personal reasons at the moment, so please forgive me if it takes a little while for me to respond (and in the mean time note that I have read stuff so they’re not being ignored) 🙂
Great points both and I agree that the kind of tradeoff/scenario described by @EJT and @bruce in his comment are the strongest/best/most important objections to my view (and the thing most likely to make me change my mind)
Let me just quote Bruce to get the relevant info in one place and so this comment can serve as a dual response/update. I think the fundamentals are pretty similar (between EJT and Bruce’s examples) even though the exact wording/implementation is not:
A) 70 years of non-offsettable suffering, followed by 1 trillion happy human lives and 1 trillion happy pig lives, or
B) [70 years minus 1 hour of non-offsettable suffering (NOS)], followed by 1 trillion unhappy humans who are living at barely offsettable suffering (BOS), followed by 1 trillion pig lives that are living at the BOS,
You would prefer option B here. And it’s not at all obvious to me that we should find this deal more acceptable or intuitive than what I understand is basically an extreme form of the Very Repugnant Conclusion, and I’m not sure you’ve made a compelling case for this, or that world B contains less relevant suffering.
to which I replied:
Yeah not going to lie this is an important point, I have three semi-competing responses:
I’m much more confident about the (positive wellbeing + suffering) vs neither trade than intra-suffering trades. It sounds right that something like the tradeoff you describe follows from the most intuitive version of my model, but I’m not actually certain of this; like maybe there is a system that fits within the bounds of the thing I’m arguing for that chooses A instead of B (with no money pumps/very implausible conclusions following)
Well the question again is “what would the IHE under experiential totalization do?” Insofar as the answer is “A”, I endorse that. I want to lean on this type of thinking much more strongly than hyper-systematic quasi-formal inferences about what indirectly follows from my thesis.
I think it’s possible that the answer is just B because BOS is just radically qualitatively different from NOS.
Maybe most importantly I (tentatively?) object to the term “barely” here because under the asymptotic model I suggest, the value of subtracting arbitrarily small amount of suffering instrument from the NOS state results in no change in moral value at all because (to quote myself again) “Working in the extended reals, this is left-continuous: ”
So in order to get BOS, we need to remove something larger than , and now it’s a quasi-empirical question of how different that actually feels from the inside. Plausibly the answer is that “BOS” (scare quotes) doesn’t actually feel “barely” different—it feels extremely and categorically different
Consider “which of these responses if any is correct” a bit of an open question for me.
Plausibly I should have figured this out before writing/publishing my piece but I’ve updated nontrivially (though certainly not all the way) towards just being wrong on the metaphysical claim.
This is in part because after thinking some more since my reply to Bruce (and chatting with some LLMs), I’ve updated away from my points (1) and (2) above.
I am still struggling with (3) both at:
the conceptual level of whether it could be the case that there are fundamental qualitative discontinuities corresponding to the asymptote location at arbitrarily small but not infinitesimal (!) changes in i_s; and
the quasi-empirical level of whether that’s actually how things are
Mostly (2) though, I should add. I think (uncertain/tentative etc etc) that this is conceptually on the table.
So to respond to Ben:
I interpret OP’s point about asymptotes to mean that he indeed bites this bullet and believes that the “compensation schedule” is massively higher even when the “instrument” only feels slightly worse?
I don’t bite the bullet in the most natural reading of this, where very small changes in i_s do only result in very small changes in subjective suffering from a subjective qualitative POV. Insofar as that is conceptually and empirically correct, I (tentatively) think it’s a counterexample that more or less disproves my metaphysical claim (if true/legit).
But I feel pretty conflicted right now about whether the small but not infinitesimal change in i_s → subjectively small difference is true (again, mostly because of quasi-empirical uncertainty).
This is hard to think about largely because my model/view leaves the actual shape of the asymptote unspecified (here’s a new version of the second pic in my post), and that includes all the uncertainty associated with what instrument we are literally or conceptually talking about (since the sole criterion is that it’s monotonic)[1]
I will add that one reason I think this might be a correct “way out” is that it would just be very strange to me if “IHE preference is to refuse 70 year torture and happiness trade mentioned in post” logically entails (maybe with some extremely basic additional assumptions like transitivity) “IHE gives up divine bliss for a very small subjective amount of suffering mitigation”
I know that this could just be a failure of cognition and/or imagination on my part. Tbh this is really the thing that I’m trying to grok/wrestle with (as of now, like for the last day or so, not in the post)
I also know this is ~motivated reasoning, but idk I just do think it has some evidential weight. Hard to justify in explicit terms though.
I’m curious if others have different intuitions about how weird/plausible this [2] is from a very abstract POV
Mostly for fun I vibecoded an API to easily parse EA Forum posts as markdown with full comment details based on post URL (I think helpful mostly for complex/nested comment sections where basic copy and paste doesn’t work great)
I have tested it on about three posts and every possible disclaimer applies
Again I appreciate your serious engagement!
The positive argument for the metaphysical claim and the title of this piece relies (IMO) too heavily on a single thought experiment, that I don’t think supports the topline claim as written.
Not sure what you mean by the last clause, and to quote myself from above:
I don’t expect to convince all readers, but I’d be largely satisfied if someone reads this and says: “You’re right about the logic, right about the hidden premise, right about the bridge from IHE preferences to moral facts, but I would personally, both in real life and as an IHE, accept literally anything, including a lifetime of being boiled alive, for sufficient happiness afterward.”
Yeah it’s possible I should (have) emphasized this specific thesis (“IHE thought experiment, I claim, is an especially epistemically productive way of exploring that territory, and indeed for doing moral philosophy more broadly”) more as an explicit claim, distinct from the two I highlight as the organizing/motivating claims corresponding to each section. Maybe I will add a note or something about this.
I don’t have a rock solid response to the “too heavily” thing because, idk, I think the thought experiment is actually what matters and what corresponds to the true answer. And I’ll add that a background stance I have is that I’m trying to convey what think is the right answer, not only in terms of explicit conclusions but in terms of what evidence matters and such.
A) 70 years of non-offsettable suffering, followed by 1 trillion happy human lives and 1 trillion happy pig lives, or
B) [70 years minus 1 hour of non-offsettable suffering (NOS)], followed by 1 trillion unhappy humans who are living at barely offsettable suffering (BOS), followed by 1 trillion pig lives that are living at the BOSYou would prefer option B here. And it’s not at all obvious to me that we should find this deal more acceptable or intuitive than what I understand is basically an extreme form of the Very Repugnant Conclusion, and I’m not sure you’ve made a compelling case for this.
Yeah not going to lie this is an important point, I have three semi-competing responses:
I’m much more confident about the (positive wellbeing + suffering) vs neither trade than intra-suffering trades. It sounds right that something like the tradeoff you describe follows from the most intuitive version of my model, but I’m not actually certain of this; like maybe there is a system that fits within the bounds of the thing I’m arguing for that chooses A instead of B (with no money pumps/very implausible conclusions following)
Well the question again is “what would the IHE under experiential totalization do?” Insofar as the answer is “A”, I endorse that. I want to lean on this type of thinking much more strongly than hyper-systematic quasi-formal inferences about what indirectly follows from my thesis.
I think it’s possible that the answer is just B because BOS is just radically qualitatively different from NOS.
Maybe most importantly I (tentatively?) object to the term “barely” here because under the asymptotic model I suggest, the value of subtracting arbitrarily small amount of suffering instrument from the NOS state results in no change in moral value at all because (to quote myself again) “Working in the extended reals, this is left-continuous: ”
So in order to get BOS, we need to remove something larger than , and now it’s a quasi-empirical question of how different that actually feels from the inside. Plausibly the answer is that “BOS” (scare quotes) doesn’t actually feel “barely” different—it feels extremely and categorically different
Consider “which of these responses if any is correct” a bit of an open question for me.
And I’ll add that insofar as the answer is (2) and NOT 3, I’m pretty inclined to update towards “I just haven’t developed an explicit formalization that handles both the happiness trade case and the intra-suffering trade case yet” more strongly than towards “the whole thing is wrong, suffering is offsetable by positive wellbeing”—after all, I don’t think it directly follows from “IHE chooses A” that “IHE would choose the 70 years of torture.” But I could be wrong about this! I 100% genuinely think I’m literally not smart enough to intuit super confidently whether or a formalization that chooses both A and no torture exists. I will think about this more!
Thought experiment variations:
People’s intuitions about the suffering/bliss trade might reasonably change based on factors like:
Duration of suffering (70 minutes vs. 70 years vs. 70 billion years)
Whether experiences happen in series or parallel
Whether you can transfer the bliss to others
I agree (1) offers interesting variations. I do have a vague, vibey sense that one human lifetime seems like a pretty “fair” central case to start from but this is not well-justified.
I more strongly want to push back on (2) and (3) in the sense that I think parallel experience, while probably conceptually fine in principle, really greatly degrades the epistemic virtue of the thought experiment because this literally isn’t something human brains were/are designed to do or simulate. And likewise with (3), the self interest bit seems pretty epistemically important.
Threshold problem:
Formalizing where the lexical threshold sits is IMO pretty important, because there are reasonable pushbacks to both, but they feel like meaningfully different views
High threshold (e.g.,”worst torture”) leads to unintuitive package deals where you’d accept vast amounts of barely-offsettable suffering (BOS) to avoid small amounts of suffering that does cross the threshold
Low threshold (e.g., “broken hip” or “shrimp suffering”) seems like it functionally becomes negative utilitarianism
I agree it is imporant! Someone should figure out the right answer! Also in terms of practical implementation, probably better to model as a probability distribution than a single certain line.
Asymptotic compensation schedule:
The claim that compensation requirements grow asymptotically (rather than linearly, or some other way) isn’t well-justified, and doesn’t seem to meaningfully change the unintuitive nature of the tradeoffs your view is willing to endorse.
I disagree that it isn’t well-justified in principlle, but maybe I should have argued this more thoroughly. It just makes a ton of intuitive sense to me but possibly I am typical-minding. And I’m pretty sure you’re wrong about the second thing—see point 3 a few bullets up. It seems radically less plausible to me that the true nature of ethics involves discontinuous i_s vs i_h compensation schedules.
Ok lol your comment is pretty long so I think I will need to revisit the rest of it! Some vibes are likely to include:
“I literally don’t know what the threshold is. I agree it would be nice to formalize it! My uncertainty isn’t much evidence against the view as a whole”
I feel like we agree about continuity; the asymptote seems very intuitively like the most likely way to connect “paper cuts are acceptable to create in exchange for wellbeing” and “the IHE would refuse the torture trade”. I agree it’s not literally impossible that a discontinuous model is correct
I suspect this intended to be illustrative, but I would be surprised if there were many, if any standard utilitarians who would actually say that you need TREE(3)[5] flourishing human life years to offset a cluster headache lasting 1 hour, so this seems like a strawman?
Yes illustrative, I’m not trying to claim that these people actually exist. I don’t think it’s a strawman. Maybe this is a grammatical thing with my use of the word the to refer to a hypothetical person
I made an audio version:
Also: Copy to clipboard as markdown link for LLM stuff
This is coming in handy, thanks!!
Who would win, a $3T company or one guy on the EA Forum? (answer: the latter)
I genuinely respect your attempts to find out the true answer here, but from my (relatively naive—I certainly haven’t read all your stuff) POV, shouldn’t the conclusion be much closer to “nobody knows a goddamn thing, don’t spend any money till we become more confident”
Awesome!!
Assuming we’re not radically mistaken about our own subjective experience, it really seems like pleasure is good for the being experiencing it (aside from any function or causal effects it may have).
In fact, pleasure without goodness in some sense seems like an incoherent concept. If a person was to insist that they felt pleasure but in no sense was this a good thing, I would say that they are mistaken about something, whether it be the nature of their own experience or the usual meaning of words.
Some people, I think, concede the above but want to object that lower-case goodness in the sense described is distinct from some capital-G objective Goodness out there in the world.
But sentient beings are a perfectly valid element of the world/universe, and so goodness for a given being simply implies goodness at large (all else equal of course). There’s no spooky metaphysical sense in which it’s written into the stars; it is simply directly implied by the facts about what some things are like to some sentient beings.
I’d add that the above logic holds fine, and with even more rhetorical and ethical force, in the case of suffering.
Now if you accept the above, here’s a simple thought experiment: consider two states of the world, identical in every way except in world A you’re experiencing a terrible stomach ache and in world B you’re not.
The previous argument implies that there is simply more badness in world A, full stop.
Much more to be said ofc but I’ll leave it there :)
I’m continually unsure how best to label or characterize my beliefs. I recently switched from calling myself a moral realist (usually with some “but its complicated” pasted on) to an “axiological realist.”
I think some states of the world are objectively better than others, pleasure is inherently good and suffering is inherently bad, and that we can say things like “objectively it would be better to promote happiness over suffering”
But I’m not sure I see the basis for making some additional leap to genuine normativity; I don’t think things like objective ordering imply some additional property which is strongly associated with phrases like “one must” or “one should”.
Of course the label doesn’t matter a ton, but I’m curious both what people think of as the appropriate label for such a set of beliefs and what people think of it on the merits.
(For those interested, I recorded a podcast on this with @sarahhw and @AbsurdlyMax a while back)
This is incredibly good and generous of you, but also I suspect that even on purely altruistic grounds it makes more sense to save the money for yourself and become slightly less risk averse as a result?
I don’t have a good model or rigorous justification for this, just an intuition
List [not necessarily final!]
1. ClusterFree
2. Center for Reducing Suffering
3. Arthropoda Foundation
4. Shrimp Welfare Project
5. Effective Altruism Infrastructure Fund
6. Forethought Foundation
7. Wild Animal Initiative
8. Center for Wild Animal Welfare
9. Animal Welfare Fund
10. Aquatic Life Institute
11. Longview Philanthropy’s Emerging Challenges Fund
12. Legal Impact for Chickens
13. The Humane League
14. Rethink Priorities
15. Centre for Enabling EA Learning & Research
16. MATS Research
Methodology
I used AI for advice (unlike last year) with Claude-Opus-4.5 and Gemini-3-Pro-Preview. I didn’t take either of their suggested rankings of course, but they both gave me a pretty decent starting point to work with such that I’m pretty sure I’d endorse either’s list as a marginal improvement to the vote distribution
Both were prompted with a list of my values and takes in addition to roughly 200k tokens scraped from the relevant posts list.
More substantially, my final list was probably based on a combination of not-really-all-that-amazing-but-better-than-nothing heuristics: what I feel like EA at large is under-funding, what I would be most excited to see, what tentatively aligns with my literally endorsed beliefs about the nature of suffering, what cause areas make theoretical sense to be near the Pareto frontier.
Take
Honestly I think I would (will?) almost certainly adjust my list if I look(ed) into it for just a few additional hours
At some level, implicitly ranking charities [eg by donating to one and not another] is kind of an insane thing for an individual to do—not in an anti-EA way (you can do way better than vibes/guessing randomly) but in a “there must be better mechanisms/institutions for outsourcing donation advice than GiveWell and ACE and ad hoc posts/tweets/etc and it’s really hard and high stakes” way.
Like what I would love is a lineup of 10-100 very highly engaged and informed people (could create the list simply by number of endorsements/requests) who talk about their strategy and values in a couple pages and then I just defer to them (does this exist?)