Working to reduce extreme suffering for all sentient beings.
Author of Suffering-Focused Ethics: Defense and Implications; Reasoned Politics; & Essays on Suffering-Focused Ethics.
Co-founder (with Tobias Baumann) of the Center for Reducing Suffering (CRS).
To clarify, the context of the quoted remark was that, just as we can care for those we love in the face of cluelessness, we can likewise care for and benefit strangers.
Specifically in relation to this:
we still have reason to respect other values we hold dear — those that were never grounded purely in the impartial good in the first place. Integrity, care for those we love, and generally not being a jerk, for starters. Beyond that, my honest answer is: I don’t know.
I think the “other values we hold dear” can and should also include a strong focus on helping strangers as well, even if challenges from cluelessness and the like were to prove devastating to standard versions of consequentialism. I mostly responded to what I saw as an overly exclusive focus on those we love versus others.
I was simply erring on the side of being generous to the non-clueless view
Right, I suspected that — hence the remark about infinite ethics considerations counting as an additional problem to what’s addressed here. My point was that the non-clueless view addressed here (finite case) already implicitly entails scope limitations, so if one embraces that view, the question seems to be what the limitation (or discounting) in scope is, not whether there is one.
To clarify, what I object to here is not a claim like “very strong consequence-focused impartiality is most plausible all things considered”, or “alternative views also have serious problems”. What I push back against is what I see as an implied brittleness of the general project of effective altruism (broadly construed), along the lines of “it’s either very strong consequence-focused impartiality or total bust” when it comes to working on EA causes/pursuing impartial altruism in some form.
On the first point, you’re right, I should have phrased this differently: it’s not that those passages imply that impartiality entails consequentialism (“an act is right iff it brings about the best consequences”). What I should have said is that they seem to imply that impartiality at a minimum entails strong forms of consequence-focused impartiality, i.e. the impartiality component of (certain forms of) consequentialism (“impartiality entails that we account for all moral patients, and all the most significant impacts”). My point was that that’s not the case; there are forms of impartiality that don’t — both weaker consequence-focused notions of impartiality as well as more rule-based notions of impartiality (etc), and these can be relevant to, and potentially help guide, ethics in general and altruism in particular.
Can you say more why you think it’s very strong?
I think it’s an extremely strong claim both because there’s a broad set of alternative views that could potentially justify varieties of impartial altruism and work on EA causes — other than very strong forms of consequence-focused impartiality that require us to account for ~all consequences till the end of time. And the claim isn’t just that all those alternative views are somewhat implausible, but that they are all wholly implausible (as seems implied by their exclusion and dismissal in passages like “impartial altruism would lose action-guiding force”).
One could perhaps make a strong case for that claim, and maybe most readers on the EA Forum endorse that strong claim. But I think it’s an extremely strong claim nevertheless.
At a conceptual level, I think it’s worth clarifying that “impartiality” and “impartial altruism” do not imply consequentialism. For example, the following passages seem to use these terms as though they must imply consequentialism. [Edit: Rather, these passages seem to use the terms as though “impartiality” and the like must be focused on consequences.]
impartiality entails that we account for all moral patients, and all the most significant impacts we could have on them. …
Perhaps it’s simply indeterminate whether any act has better expected consequences than the alternatives. If so, impartial altruism would lose action-guiding force — not because of an exact balance among all strategies, but because of widespread indeterminacy.
Yet there are forms of impartiality and impartial altruism that are not consequentialist in nature [or focused on consequences]. For example, one can be a deontologist who applies the same principles impartially toward everyone (e.g. be an impartial judge in court, treat everyone you meet with the same respect and standards). Such impartiality does not require us to account for all the future impacts we could have on all beings. Likewise, one can be impartially altruistic in a distributive sense — e.g. distributing a given resource equally among reachable recipients — which again does not entail that we account for all future impacts.
I don’t think this is merely a conceptual point. For example, most academic philosophers, including academic moral philosophers, are not consequentialists, and I believe many of them would disagree strongly with the claim that impartiality and impartial altruism imply consequentialism.[1] Similarly, while most people responding to the EA survey of 2019 leaned toward consequentialism, it seems that around 20 percent of them leaned toward non-consequentialism, and presumably many of them would also disagree with the above-mentioned claim.
Furthermore, as hinted in another comment, I think this point matters because it seems implied a number of times in this sequence that if we can’t ground altruism in very strong forms of consequentialist impartiality, then we have no reason for being altruists and impartial altruism cannot guide us (e.g. “if my arguments hold up, our reason to work on EA causes is undermined”; “impartial altruism would lose action-guiding force”). Those claims seem to assume that all the alternatives are wholly implausible (including consequentialist views that involve weaker or time-adjusted forms of impartiality). But that would be a very strong claim.
They’d probably also take issue with defining an “impartial perspective” as one that is consequentialist: “one that gives moral weight to all consequences, no matter how distant”. That seems to define away other kinds of impartial perspectives.
“reliably” doesn’t mean “perfectly”
Right, I guess within my intuitive conceptions and associations, it’s more like a spectrum, with “perfectly” being the very strongest, “reliably” being somewhere in between, and something like “the tiniest bit better than chance” being the weakest. I suspect many would endorse ~the latter formulation without endorsing anything quite as strong as “reliably”.
To be clear, I don’t think this is a matter of outright misrepresenting others’ views; I just suspect that many, maybe most, of those who hold a contrary view would say that those specific descriptions are not particularly faithful or accurate framings of their views, even if certain sections do frame and address things differently.
Yeah, my basic point was that just as I don’t think we need to ground a value like “caring for those we love” in whether it has the best consequences across all time and space, I think the same applies to many other instances of caring for and helping individuals — not just those we love.
For example, if we walk past a complete stranger who is enduring torment and is in need of urgent help, we would rightly take action to help this person, even if we cannot say whether this action reduces total suffering or otherwise improves the world overall. I think that’s a reasonable practical stance, and I think the spirit of this stance applies to many ways in which we can and do benefit strangers, not just to rare emergencies.
In other words, I was just trying to say that when it comes to reasonable values aimed at helping others, I don’t think it’s a case of “it must be grounded in strong impartiality or bust”. Descriptively, I don’t think that reflects virtually anyone’s actual values or revealed preferences, and I don’t think it’s reasonable from a prescriptive perspective either (e.g. I don’t think it’s reasonable or defensible to abstain from helping a tormented stranger based on cluelessness about the large-scale consequences).
Why would sentient beings’ interests matter less intrinsically when those beings are more distant or harder to precisely foresee?
I agree with that sentiment :) But I don’t think one would be committed to saying that distant beings’ interests matter less intrinsically if one “practically cares/focuses” disproportionally on beings who are in some sense closer to us (e.g. as a kind of mid-level normative principle or stance). The latter view might simply reflect the fact that we inhabit a particular place in time and space, and that we can plausibly better help beings in our vicinity (e.g. the next few thousands of years) compared to those who might exist very far away (e.g. beyond a trillion years from now), without there being any sharp cut-off between our potential to help them.
FWIW, I don’t think it’s ad hoc or unmotivated. As an extreme example, one might consider a planet with sentient life that theoretically lies just inside our future light cone from time t_now, such that if we travelled out there today at the theoretical maximum speed, then we, or meaningful signals, could reach them just before cosmic expansion makes any further reach impossible. In theory, we could influence them, and in some sense merely wagging a finger right now has a theoretical influence on them. Yet it nevertheless seems to me quite defensible to practically disregard (or near-totally disregard, à la asymptotic discount) these effects given how remote they are (assuming a CDT framework).
Perhaps such a position can be viewed from the lens of an “applicability domain”: to a first approximation, the ideal of total impartiality is plausibly “practically morally applicable” on all of Earth and on and somewhat beyond our usual timescales. And we are right to strongly endorse it at this unusually large scale (i.e. unusual relative to prevailing values). But it also seems plausible that its applicability gradually breaks down when we approach extreme values.
Indeed, bracketing off “infinite ethics shenanigans” could be seen as an implicit acknowledgment of such a de-facto breakdown or boundary in the practical scope of impartiality. After all, there is a non-zero probability of an infinite future with sentient life, even if that’s not what our current cosmological models suggest (cf. Schwitzgebel’s Washout Argument Against Longtermism). Thus, it seems that if we limit infinite outcomes from dominating everything, we have already set some kind of practical boundary (even if it’s a practical boundary of asymptotic convergence toward zero across an in-theory infinite scope). If so, it seems that the question is to clarify the nature and scope of that practical boundary, not whether it’s there or not.
One might then say that infinite ethics considerations indeed count as an additional, perhaps also devastating challenge to any form of impartial altruism. But in that case, the core objection reduces to a fairly familiar objection about problems with infinities. If we make an alternative case, in which we assume that infinities can be set aside or practically limited, then it seems we have already de facto assumed some practical boundary.
I use “impartiality” loosely, in the sense in the first sentence of the intro: “gives moral weight to all consequences, no matter how distant”.
Thanks for clarifying. :)
How about views that gradually discount at the normative level based on temporal distance, like or so? They would give weight to consequences no matter how distant, and still give non-trivial weight to fairly distant consequences (by ordinary standards), yet the weight would go to zero as the distance grows. If normative neartermism is largely immune to your arguments, might such “medium-termist” views largely withstand them as well?
(FWIW, I think views of that kind might actually be reasonable, or at least deserve some weight, in terms of what one practically cares about and focuses on — in part for the very reasons you raise.)
I meant “the reason to work on such causes that my target audience actually endorses.”
I suspect there are many people in your target audience who don’t exclusively endorse, or strictly need to rely on, the views you critique as their reason to work on EA causes (I guess I’m among them).
Toward the very end, you write:
“But what should we do, then?” Well, we still have reason to respect other values we hold dear — those that were never grounded purely in the impartial good in the first place. Integrity, care for those we love, and generally not being a jerk, for starters. Beyond that, my honest answer is: I don’t know.
You obviously don’t exclude the following, but I would strongly hope that — beyond just integrity, care for those we love, and not being a jerk — we can also at a minimum endorse a commitment to reducing overt and gratuitous suffering taking place around us, even if it might not be the single best thing we can do from the perspective of perfect impartiality across all space and time. This value seems to me to be on a similarly strong footing as the other three you mention, and it doesn’t seem like it stands or falls with perfect [or otherwise very strong] cosmic impartiality. I suspect you agree with its inclusion, but I feel like it deserves emphasis in its own right.
Relatedly, in response to this:
Ask yourself: Does “this strategy seems good when I assume away my epistemic limitations” have the deep moral urgency that drew you to EA in the first place?
I would say “yes”, e.g. if I replace “this strategy” with something like “reducing intense suffering around me seems good [even] when I assume away my epistemic limitations [about long-term cosmic impacts]”. That does at least carry much of the deep moral urgency that motivates me. I mean, just as I can care for those I love without needing to ground it in perfect cosmic impartiality, I can also seek to reduce the suffering of other sentient beings without needing to rely on a maximally impartial perspective.
I should probably have made it more clear that this isn’t an objection, and maybe not even much of a substantive point, but more just a remark on something that stood out to me while reading, namely that the views critiqued often seemed phrased in much stronger terms than what people with competing views would necessarily agree with.
Some of the examples that stood out were those I included in quotes above.
You write the following in the first post in the sequence (I comment on it here because it relates closely to similar remarks in this post):
if my arguments hold up, our reason to work on EA causes is undermined.
This claim seems to implicitly assume that perfect impartiality [edit: or very strong forms of impartiality] across all space and time is the only reason or grounding we could have for working on EA causes. But that’s hardly the case — there are countless alternative reasons or moral stances that could ground support for EA (or work on EA causes). For example, one could be normatively near-termist, or one could gradually discount the interests of beings depending on how distant they seem to be from our potential to predictably help them (and one could then still aim to be effective and indeed impartial within the parameters of those frameworks).
Of course, one might disagree with views that aren’t perfectly impartial, and argue that they’re implausible, but that’s different from saying that they can’t coherently ground work on EA causes. (And indeed, the arguments raised in this series could be viewed as lending some support to such alternative views, or at least give us reason to be more curious about them.)
The first thought I have is mostly an impression or something that stood out to me: it seems to me like the word choices here sometimes don’t quite reflect the point being made or the full range of views being critiqued, arguably including the strongest competing views.
For example, when talking about heuristics that are supposed to be “robust”, or strategies we can “reliably intervene on”, or whether we can “reliably weigh up” relevant effects, etc, it seems to me that these word choices convey something much stronger than what would necessarily be endorsed by weaker and arguably more defensible types of views that differ from yours.
After all, views different from yours might agree that no heuristics are outright robust and that we can’t reliably weigh up the relevant effects, while nevertheless holding something weaker, like that some heuristics are slightly better than others in expectation, or that we in general can do meaningfully (or even just marginally) better than nothing.
Of course, I get that language is always a bit vague, and I’m sure you meant to include a very broad range of views, including very “weak” ones. But at least in terms of how I read it, the language often seemed to invoke a much stronger and more narrow set of views than necessary.
Kudos for writing up this series, Anthony! :) It’s not easy to face deeply unpleasant and inconvenient (potential) conclusions, let alone to stare them down and write about them.
I have a lot of thoughts on what you write, but for now I’ll share just three of them (in separate comments).
This is a very loose idea, based on observations like these:
We have ongoing geopolitical tensions (e.g. China-US, China-Taiwan, Russia-Ukraine) and a lot of resources and attention spent on those.
We have (increasing?) risks from emerging technology that potentially threaten everyone. It’s difficult to estimate the risk levels, but there seems to be an emerging consensus that we are on a reckless path, even from perspectives concerned purely with individual or national self-interest.
The project would essentially seek to make a clear case for broad cooperation toward avoiding widely agreed-upon bad outcomes from emerging technologies — outcomes that are in nobody’s interest. The work could, among other things, consist in reaching out to key diplomats as well as doing high-visibility public outreach that emphasizes cooperation as key to addressing risks from emerging technologies.
Reasons it might be worth pursuing:
The degree of cooperation between major powers, especially wrt tech development, is plausibly a critical factor in how well the future will go. Even marginal improvements might be significant.
A strong self-interested case can seemingly be made for increasing cooperation, but a problem might be its relatively low salience as well as primitive status and pride psychology preventing this case from being acted on.
Even if the case is fairly compelling to people, other motivations might nevertheless feel more compelling and motivating; slight pushes in terms of how salient certain considerations are, both in the minds of the public and leaders, could potentially tip the scales in terms of which paths end up being pursued.
The broader goal seems quite commonsensical and like something few people would outright oppose (though see the counter-considerations below).
The work might act as a lever or catalyst of sorts: one can make compelling arguments regarding specific technological risks, but if those arguments aren’t explicitly tied to clear and practical upshots about international cooperation (in a memorable way that doesn’t drown in other info), those arguments might not be so effective and they might not lead to much work getting done on those risks. Conversely, if cooperation levels are successfully raised, this could help open the floodgates of power and resources held by major governments to support serious action.
Indeed, a key aim of a megaproject like this could be to unleash a mega-megaproject that has the resources of the major world powers behind it.
Early exploration and work on how to do successful outreach could be key to its success (e.g. identifying how certain arguments might be misconstrued and generally identifying and avoiding communication pitfalls that might be unique to this area).
It seems like it could be quite tractable if one has the right people working on it (the tractability is obviously uncertain, though maybe it’s worth a bet anyway).
There might be a big difference in effectiveness between mentioning cooperation as (what can seem like) a sort of afterthought versus advocating for it in a very focused way. I think a high level of focus could be a unique strength of a project like this.
Reasons it might not be worth pursuing (I’m not convinced that it in fact is):
There might be better opportunities to improve the future.
It could backfire if done poorly.
Greater international cooperation could be harmful (e.g. it could cement harmful human values).
Relatedly, a premature emphasis on cooperation in our current geopolitical environment could reduce pressure on authoritarian governments and may inadvertently prevent a more democratic world that might have been better able to address these risks (though promoting cooperation to address technological risks could also have a democratizing effect, it’s quite unclear).
Whether it’s framed as a megaproject or not (presumably one would start out smaller in any case), it seems to me that the idea of a focused and ambitious project in this ballpark is at least worth exploring further. My impression is that some people are already working on this, but perhaps not in a very public and visible way — though maybe I’ve just missed it?
(Added a graph that visualizes the naive projection discussed above.)
The Codeforces Elo progression from o1-mini to o3-mini was around 400 points (with compute costs held constant). Similarly, the Elo jumps from 4o (~800) to o1-preview (~1250) to o1-mini (~1650) were also each around 400 points (the compute costs of 4o appear similar to those of o1-mini, while they’re higher for o1-preview).
People from OpenAI report that o4 is now being trained and that training runs take around three months in the current “reasoning paradigm”. So if we were to engage in naive projection, we might project a continued ~400 point Codeforces progression every three months.
Below is a naive such projection for the o1-mini cost range, with the dates referring to when model scores are announced (not when the models are released).
March 2025 (March 14th?): o4 ~2400
June 2025: o5 ~2800
September 2025: o6 ~3200
December 2025: o7 ~3600
If high compute adds around 700 Elo points for full o7 (as it does for o3), this would give full o7 a superhuman score of ~4300
March 2026: o8 ~4000 (a score only ever achieved by two people)
June 2026: o9 ~4400 (superhuman level for cheap)
Part of the motivation for making such a naive projection is that it can provide a salient yardstick to hold future progress up against, to notice whether progress on this benchmark is slowing down, keeping pace, or accelerating.
Additionally, as further motivation, one can note that there is some precedent for Elo scores improving linearly over time in other domains, e.g. in chess:
Likewise, while they’re more subjective, Elo scores on the LLM leaderboard also appear to have increased fairly consistently by an average of ~20 points per month over the last year (the trend has continued beyond the graph below; the current top 10 average is at the ~1360 level one would have predicted based on a naive extrapolation of the post-2023-11 trendline below):
This is what I meant:
it seems to me like a striking … kind of coincidence to end at exactly — or indistinguishably close to — … any position of complete agnosticism
That is, I think it tends to apply to complete and perfect agnosticism in general, even if one doesn’t frame or formulate things in terms of 50⁄50 or the like. (Edit: But to clarify, I think it’s less striking the less one has thought about a given choice and the less the options under consideration differ in character; so I think there are many situations in which practically complete agnosticism is reasonable.)
To clarify, the general approach outlined here doesn’t rest on the use of discount rates — that’s just a simple and illustrative example of scope-restriction.