Subtitle: Systemic change or high confidence? Pick one.
Folks used to criticize Effective Altruism for being insufficiently open to (presumed) directionally helpful but hard-to-measure efforts to address large-scale problems at their root. While the charge strikes me as misdirected,[1] I certainly agree with the underlying principle—also known as “hits-based giving”—that we should be willing to consider hard-to-quantify longshots (when supported by sufficiently good reasons—reasons that may be much more nebulous and contestable than the results of a randomized controlled trial).
The funny thing is that once Effective Altruists started pursuing this strategy more prominently—investing more in speculative “longtermist” projects like AI safety and fancy conference venues from which they might better influence policymakers—they were met with a chorus of critics lamenting their betrayal of “old-school EA values” (personal asceticism and a strong focus on reliably helping the global poor) in favor of more speculative strategies.
It’s worth noting that these two popular objections are in tension. If you’re unwilling to countenance uncertainty or “speculation” (perhaps because you discount ungrounded expected value estimates as too easily biased by personal interests) then you’re committed to embracing some form of measurability bias: prioritizing rigor over reach. Conversely, if you’re optimistic about the prospects of pursuing contestable “systemic change” to radically improve the world, you can hardly rule out a priori that influencing the trajectory of what is plausibly the most transformative technology ever invented would be a reasonable way to implement that strategy! It’s obviously highly uncertain, but that’s the tradeoff one accepts when pursuing higher potential impact: prioritizing reach over rigor.
Of course, one could always try to argue on the merits that standard EA views on both global poverty and AI safety are empirically mistaken. That’s fine. All I’m saying here is that one had better not endorse higher-level heuristics, applied prior to any engagement with the specific details, that rule out both rigorously-supported global health charities (simply for being insufficiently ambitious) and speculative longtermist longshots (simply for being insufficiently rigorous). After all, if you prioritize rigor, the prima facie case for global health charities is going to be hard to beat. And if you prioritize reach (potential impact), nobody else is even in the same ballpark as the longtermists. Many worry about the potential for bias in longtermists’ judgments, but does anyone seriously expect radical politics to be less subject to bias?
I think it will be difficult to articulate a principled case for thinking that one’s preferred politics is a better bet in expectation for improving the world than both the neartermist and longtermist wings of Effective Altruism. The odds of your individual political actions (voting, protesting, etc.) making a significant difference are comparable to those from contributing to longtermist projects,[2] but the potential benefits are vastly lower, and the odds of misjudgment and proving counterproductive strike me as higher, if anything. (The risks will depend on the precise details of your preferred politics; the more radical your proposed economic overhauls, the more I worry about them proving counterproductive if successfully implemented.)
The upshot: I’d like to see more of the rigor-seeking anti-longtermists taking the side of the neartermist EAs against their speculative socialist critics. And I’d like to see fans of ambitious “systemic change” acknowledge that the longtermist branch of EA is following the right methodology in principle; the disagreement is just about which systemic changes are most promising. It’s worth figuring out where you stand, in principle, on the question of how to navigate rigor-reach tradeoffs: Pick your poison!
- ^
EA principles obviously entail support for “systemic changes” if the evidence supports the claim that they have higher expected value than alternative uses of the resources in question. Open Philanthropy (now renamed Coefficient Giving) did fund various criminal justice reform efforts, for example, which was probably a poor use of $200M. EA-funded animal welfare work is arguably “systemic”—e.g. corporate cage-free campaigns and research into cultivated meat that could eventually displace animal agriculture—and seems fairly promising to me.
My sense is that many people pressing the “systemic change” critique were misapplying a procedural gloss to what was really just a first-order political complaint that EAs don’t sufficiently share their enthusiasm for socialism in particular. (A bit like how deontologists misleadingly gloss their substantive anti-aggregative views as procedurally more respectful of “the separateness of persons”. There’s simply no basis for their claiming sole ownership of the procedural virtue—if anything, I’d argue the opposite.)
- ^
A recent tweet seemed to assume it was unacceptable for an individual’s longtermist project to have a “one in a million” chance of success. But have you compared the odds of placing a tie-breaking vote in a presidential election? Both have the potential to be extremely good bets, in terms of impartial expected value, despite the low absolute odds. But the potential longtermist payoff is also many, many orders of magnitude greater.
The key issue, I think, is ensuring that the potential benefit isn’t outweighed by the risk of causing immense negative impact. Many have argued that early AI safety concerns had the unfortunate effect of accelerating AI development, for example. But I’d want to see more argument for expecting today’s marginal AI safety efforts to have the same effect: the race is already underway, and would hardly stop just because AI safety efforts do. (I’m open to further argument; just explaining why I’m not moved by the argument from crude induction.)
I’m okay with balancing trade-offs between rigor and reach when the rigor is reasonably high. But it’s important to remember that rigor and effectiveness are correlated with each other.
When it comes to longtermism, as in trying to affect the future a thousand the rigor is essentially non-existent, and precisely because of that, I doubt there is any reach either.
Good framing!
A framing I had in a draft post was whether, in the ITN framework, to focus mostly on (disclaimer: very rough):
Scale: this is the AI safety approach. “This could be really big (and is also neglected), and therefore we should see whether it’s tractable—even if it has a minuscule chance of being tractable, it’s worth pursuing”.
Tractability: this is more the OG-global-health approach: “We can demonstrate that our intervention makes measurable progress on the issue. Other more neglected or larger-scale problems cannot do this.”
Neglectedness: this is probably the least common proxy, but you see it in animal welfare sometimes: “No one is working on this species, therefore someone should start a project”.
I do think rigor-reach / tractability-scale is the main axis, and that neglectedness is more minor. I think this can lead to unresolvable disagreements on what interventions should be pursued.
On AI safety harms beyond historical induction, there was a recent short post on ways current AI safety efforts could be net-negative.
Hi Richard.
Are you confident the longtermist interventions you have in mind, like work on AI safety, are more cost-effective accounting for longterm effects than global health interventions? Decreasing the risk of human extinction may be less cost-effective than global health interventions accounting for longterm effects hold under standard population models. So it is unclear to me whether there is the rigor-reach dilemma you describe.
I can even see global health interventions decreasing nearterm extinction risk more cost-effectively than AI safety interventions. I am not aware of any quantitative modelling comparing both options.
I don’t think you can extrapolate standard population models past the invention of artificial wombs and robo-nannies. The “Recovery” premise is much more plausible if creating new lives is less personally costly.
That said, I appreciate the reminder that nearterm life-saving plausibly has positive longterm flow-through effects (whether via population or economic growth). So the charge of “insufficient ambition” may partly reflect insufficient attention on the part of the critics to these secondary effects.
Why would artificial wombs and robo-nannies have very different effects from other innovations that have decreased the time cost of having children (for example, washing machines), or increased the income of prospective parents (for example, increasing income of women)? A lower cost of having children as a fraction of available income would increase fertility if the cost of everything else remained the same, but alternatives would become cheaper too relative to available income. It is unclear to me whether your point strengthens or weakens the conclusions of Alexandrie and Eden (2025) under the population models they analysed.
“All I’m saying here is that one had better not endorse higher-level heuristics, applied prior to any engagement with the specific details, that rule out both rigorously-supported global health charities (simply for being insufficiently ambitious) and speculative longtermist longshots (simply for being insufficiently rigorous).”
Both of these criticisms are valid, and not necessarily in tension with one another. Neartermist work can be too myopic and insufficiently ambitious, and longtermist work can be terribly weak in rigor and evidence.
Perhaps you are focused less on the critique and are rather pointing out who uses them for what ends when you say “applied prior to any engagement with the specific details”. These critiques can stand on their own, even if they are often leveled insincerely to justify some alternative preferred social or political agenda.
Right, one can always argue the specific details; here I’m talking about sweeping heuristics that are offered in place of engagement with the first-order details, i.e. assuming both that all global health charities are insufficiently ambitious and all longtermist work is insufficiently rigorous to even be worth considering.
I think your factional infighting angle isn’t really substantiated. Many rigor-seeking anti-longtermists seem to believe that risks from AI are overblown (I disagree with this) and their rigor-related critiques are connected to this. I don’t see why they have any obligation to side against EA’s socialist critics, for two reasons:
They could believe that there is rigorous evidence for socialism or some version of the socialist critique of EA.
They could believe that AI risk has more traction within EA compared to socialists critiques of EA (a correct assessment) and therefore prioritize critiquing AI risk advocates rather than socialists.
Personally, I’d like to see EAs, rationalists, and critics of both of these movements arguing the actual fucking point more, instead of making meta-arguments about why the people they disagree with are violating some newly invented standard. AI risk advocates don’t need to convince rigor-seeking anti-longtermists that actually AI risk is the most scientifically rigorous conclusion to have ever existed in order to decide that they are concerned about AI risk. Nor do rigor-seeking anti-longtermists need to stamp out all socialist critics of EA before critiquing EA for a lack of rigor. We can all just argue the actual fucking point instead.
I agree that the first-order arguments are what ultimately matter. But I also think it can be helpful to employ some meta/”outside view” tests of consistency, etc., especially since the way that most people actually come to their conclusions (afaict) rests much more on rough trust heuristics than on assessing complex issues on their merits. (Of course, if you’re sick of such discussions then you needn’t engage in this one!)
Great observation. Perhaps there’s an engaging way of explaining this to the public using a series of dilemmas or a story? Just an area one could explore.