If you value future people, why do you consider near term effects?
Edit 2024: There’s plenty here I would no longer endorse — the piece is overconfident in some places, and confused in others. If you read it, consider also reading ‘When to get off the train to crazy town?’
[Nothing here is original, I’ve just combined some standard EA arguments all in one place]
Introduction
I’m confused about why EAs who place non-negligible value on future people justify the effectiveness of interventions by the direct effects of those interventions. By direct effects I mean the kinds of effects that are investigated by GiveWell, Animal Charity Evaluators, and Charity Entrepreneurship. I mean this in contrast to focusing on the effects of an intervention on the long-term future as investigated by places like Open Phil, the Global Priorities Institute, and the Future of Humanity Institute.
This post lays out my current understanding of the problem so that I can find out the bits I’m missing or not understanding properly. I think I’m probably wrong about something because plenty of smart, considerate people disagree with me. Also, to clarify, there are people I admire who choose to work on or donate to near-term causes.
Section one states the problem of cluelessness (for a richer treatment read this: Cluelessness, Hilary Greaves) and explains why we can’t ignore the long-term effects of interventions.
Section two points at some implications of this for people focussed on traditionally near-term causes like mental health, animal welfare, and global poverty. I think these causes all seem pressing. I think that they are long-term problems (ie. poverty or factory farms now are just as bad as poverty or factory farms in 1000 years) and that it makes sense to prioritise the interventions that have the best long-term effects on these causes.
Section three tries to come up with objections to my view, and respond to them.
1. Cluelessness and Long-term Effects
Simple cluelessness
All actions we take have huge effects on the future. One way of seeing this is by considering identity-altering actions. Imagine that I pass my friend on the street and I stop to chat. She and I will now be on a different trajectory than we would have been otherwise. We will interact with different people, at a different time, in a different place, or in a different way than if we hadn’t paused. This will eventually change the circumstances of a conception event such that a different person will now be born because we paused to speak on the street. Now, when the person who is conceived takes actions, I will be causally responsible for those actions and their effects. I am also causally responsible for all the effects flowing from those effects.
This is an example of simple cluelessness, which I don’t think is problematic. In the above example, I have no reason to believe that the many consequences that would follow from pausing would be better than the many consequences that follow from not pausing. I have evidential symmetry between the two following claims:
Pausing to chat would have catastrophic effects for humanity
Not pausing to chat would have catastrophic effects for humanity
And similarly, I have evidential symmetry between the two following claims:
Pausing to chat would have miraculous effects for humanity
Not pausing to chat would have miraculous effects for humanity
(I’m assuming there’s nothing particularly special about this chat—eg. we’re not chatting about starting a nuclear war or influencing AI policy.)
And for all resulting states of the world between catastrophe and miracle. I have evidential symmetry between act-consequence pairs. By evidential symmetry between two actions, I mean that, though massive value or disvalue could come from a given action, these effects could equally easily, and in precisely analogous ways, result from the relevant alternative actions. In the previous scenario, I assume that each of the possible people that will be born are as likely as each other to be the next Norman Borlaug. And each of the possible people are as likely as each other to be the next Joseph Stalin.
So this situation isn’t problematic; the possible effects, though they are huge, cancel out in my expected value estimate.
Complex cluelessness
Cluelessness is problematic in situations where we do not have evidential symmetry. For a pair of actions (act one and act two), we have complex cluelessness when:
We have some reasons to think that the effects of act one would systematically tend to be substantially better than those of act two;
We have some reasons to think that the effects of act two would systematically tend to be substantially better than those of act one;
It is unclear how to weigh up these reasons against one another. (Here there is no evidential symmetry between act-consequence pairs. You have no EV estimate for taking one of the actions over another.)
(An explanation of what is meant by ‘systematically’ can be found in section 5 of Cluelessness, Hilary Greaves)
For example, we have some reasons to think that the long-term effects of a marginally higher economic growth rate would be good—for example, via driving more patient and pro-social attitudes. This would mean that taking action to increase economic growth could have much better effects than not taking the action. We have some reasons to think that the long-term effects of a marginally higher economic growth rate would be bad—for example, via increased carbon emissions leading to climate change. This would mean that not taking the action that increases economic growth could be a much better idea. It’s not immediately obvious that one of these is better than the other, but we also can’t say they have equal expected value. That would need either evidential symmetry, or a very detailed EV estimate. (Evidential symmetry here would be something like: every way a higher growth rate would be good is also an equally plausibly reason it would be bad eg. increased emissions are equally likely to be good as they are to be bad.)
I think that complex cluelessness implies we should be very skeptical of interventions whose claim to cost-effectiveness is through their direct, proximate effects. As has been well argued elsewhere, the long-term effects of these actions probably dominate. But we don’t know what the long-term effects of many interventions are or just how good or bad they will be.
Actions we take today have indirect long-term effects, and they seem to dominate over the direct near-term effects. Unless we have evidential symmetry we cannot ignore these long-term effects. So it seems to be that, if we care about future people, we’ll have to justify our interventions via their long-term effects, not their proximate ones.
2. Direct Effects
What position are we in?
We think our actions now have these huge effects on the future
These effects seem morally relevant (again, assuming you value the future)
These long-term effects dominate the proximate ones
We’re trying to find the actions that we have good reason to believe are the most cost-effective at improving the world (because we’re trying to improve the world as much as we can, and we have limited resources)
The direct approach (eg. looking at QALYs or deaths averted) doesn’t look at all the effects of our actions. In particular, the biggest effects (the long-term ones) are ignored. I think this means we shouldn’t use this approach to determine which interventions are most cost-effective. To me, it makes more sense, even if you’re focused on traditionally near-termist causes like mental health, animal welfare, and global poverty, to evaluate interventions based on their long-term effects.
An analogy:
(Don’t worry—I’m not going to start proving things by analogy! This is just an intuition pump and I’m aware that it breaks down.)
Imagine a hotel with 1,000 single-occupant rooms. You are in the control room of the hotel and you can push different buttons that will do different things to the hotel occupants. Every button does something to every person, but you don’t know exactly what. You think some buttons cause bliss, or torture, or death for people in particular rooms. For most rooms, it’s very hard (but somewhat tractable) to get data on how the inhabitants feel about you pushing particular buttons. Fortunately, for room #327, it’s much easier to find out how pushing different buttons affects the occupant. If you care about every occupant should you:
Get a bunch of data on how particular buttons affect room #327 and then press the buttons that you think are best for that one person
or
Put your resources into estimating how different buttons affect all the rooms?
The direct approach is analogous to getting a bunch of data on how particular buttons affect room #327 and then pressing the buttons that you think are best for that one person.
This seems weird to me if you know that the buttons affect all 1,000 rooms. You might know that a button has good effects for room #327, but it could be torture for everyone in all the other rooms. Or there might be a button that doesn’t affect room #327 much but produces waves of meaningful bliss for everyone else.
My intuition here is that putting a lot of effort into finding out how different buttons affect all the rooms makes more sense. Then you can push the button that’s your best guess at being best for all 1,000 people in aggregate. Sure, it’s really hard to get data on how everyone is affected but that doesn’t mean we can just ignore it—it’s the most important consideration for which button to press.
Global Poverty
(Relevant post: Growth and the case against randomista development)
Under a long-termist framework, it’s possible we could weigh the effects of work on different causes and decide that global poverty was the best thing to be working on. It could further be the case that current GiveWell recommended charities are the best way to go. But that whole analysis would have to be justified by the effects on future people via flow-through effects rather than effects on something like present-day QALYs.
For example, we might decide that marginally increasing economic growth isn’t too dangerous after all (e.g. because the negative effects of the poor meat eater problem, increased emissions, or increased anthropogenic existential risk are outweighed by the benefits). We might then take cost-effective actions to accelerate growth, perhaps focusing on poor countries. These might be things like charter cities or macroeconomic stabilisation, or something else we haven’t considered.
I’m confused about why some EAs who value the future and are interested in global poverty seem to prefer AMF, SCI, or GiveDirectly over these things (side note: even if you prioritise these, it’s really worth considering investing now so you can give more later). The way the EA community got to care about AMF was by analysis of a small subset of AMF’s effects. AMF has far more effects than those that are measured so, under this longtermist framework, we don’t have any evidence of the cost-effectiveness of AMF’s actions.
I think there might be good reasons to think that present day QALYs or deaths averted are good correlates of total (long-term) value—perhaps because of flow through effects. But I don’t think this is obvious at all, and I think the burden of proof is on those claiming the correlation between near-term QALYs and long-term value is strong. I don’t regularly see people justifying global poverty interventions based on their flow through effects, and I’d love to see more of this (though, of course, it’s very difficult).
An interesting point here is that, if it were true that the most effective global poverty interventions turned out to be broad growth-boosting interventions, the EA position would come a little closer to the mainstream development economics view—which I think is reassuring.
Animal Welfare
(Relevant post: Should Longtermists Mostly Think About Animals?)
(I don’t know much about animal welfare interventions at all, so expect I’m missing something here.)
People who value future nonhuman animals might achieve their goals better if they asked more questions like:
‘How can we increase the probability of factory farming ending in the next 100 years?’
‘How can we reduce the probability that factory farming continues for thousands of years?’
‘How can we reduce the probability of humanity spreading wild animal suffering across the cosmos?’
I think questions like the following seem valuable only insofar as they contribute to the first kind of question:
‘How can we avert the most present-day suffering for a given amount of money?’
‘How can we make present-day factory farmed animals suffer less?’
Again, it could be that ACE-recommended charities are the best place to donate and that current strategies (like corporate campaigns or working on clean meat) are the best kinds of direct work available. But the most effective interventions are the ones that are most effective across all time, not just the next few years or decades. Why? Because the long-term effects of animal welfare interventions will vastly dominate the near-term effects of those interventions.
Mental Health
Similarly for mental health, I’d argue that we don’t want to focus on buying QALYs now—we want to do long-lasting things like answering foundational questions, building an effectiveness-minded mental health field, and setting up institutions that will improve long-term mental health. For example, I’m excited about the research that HLI and QRI are doing. Of course, we need to roll out proposed interventions once they come around. We’ll need to test them and this will involve measurement of direct effects. But the primary value of this exploration is in the information value, and the field-building effects, not the direct welfare benefits.
Comparison to X-risk reduction
This focus on long-term field-building and trajectory change is different to biorisk, or short-timeline AI safety. For these two causes, there is risk of lock-in of some very bad state (extinction, or worse) sometime soon. This means it’s more urgent to do direct work right now to avoid the lock-in.
You could push back on this distinction by saying that there is risk of astronomical poverty lock-in or animal suffering lock-in in the next 200 years. Perhaps we will start space colonisation in that time and then fall into some weird Malthusian-style situation later on (see This is the Dream Time, and Potatonium (though the situation described here might be a good one)). Or perhaps we’ll expand to other planets and bring wild animals or factory farms with us. These things are concerning but they don’t seem to obviously point to donating to ACE or GiveWell charities as the solution.
3. Objections
[The point of this post is that I don’t adequately understand the best arguments against my view. So my understanding of the objections to my view is obviously limited]
Near-term work is more certain
Objection: The route to value of some types of long-term work is highly uncertain, with very small probabilities of very large payoffs. If I want to be sure that I do at least some good, maybe I should prioritise more certain near-term work.
Response: If we care about all the effects of our actions, it’s not clear that near-term interventions are any less speculative than long-term interventions. This is because of the dominating but uncertain long-term effects of near-term interventions.
Near-term work is more evidence-based
Objection: For any action, it’s usually much harder to get evidence about it’s long-term effects than it’s near-term effects. So, given that we are using evidence to improve the world, maybe we should focus on the effects we can measure. It could be much easier to make a dent in near-term problems because we have much more evidence about them.
Response: It’s true that we don’t have much evidence about the long-term effects of our actions. But if we think those effects are morally relevant, we cannot ignore them (this is complex cluelessness, not simple). Rather, we should invest resources in getting more evidence about those effects. Unfortunately, this evidence isn’t going to be through randomised controlled trials (RCTs) or anything as rigorous as that. I agree that longtermism presents a huge epistemic challenge and, if we want to help people as much as possible, we have to deeply understand the past, and build excellent models of the future. We’ll need to get much better at rationality, forecasting, and generally understanding the world to do this.
[This is related, particularly the introduction: Reality is often underpowered]
Long-term work is subject to bias
Objection: Because the evidence we have about long-term effects is weak, there is much more weight placed on subjective judgements and expert opinion rather than RCTs or other data. In these situations, we might expect our cause prioritisation to be tracking the wrong thing—like the biases, interests, or preferences of people in the community. For example, maybe part of the reason the EA community values MIRI is because of Elieizer’s idiosyncrasies. In contrast, the EA community might value AMF because of impartial, dispassionate analysis.
Response: I think this is a good point, and something to be aware of. To me, it seems to point to doing better analysis of long-term effects, rather than to ignoring long-term effects. I’m not sure anyone uses this objection, but I’d be interested to what such people thought about the effect size of bias compared to the effect size of working on long-term causes.
If we have any effectiveness estimates at all, they are for near-term work
Objection: If we can’t get effectiveness estimates of something as measurable as AMF, how could we ever get estimates of intangible long-term effects or speculative interventions?
Response: It’s true we don’t have robust cost-effectiveness estimates for long-term interventions in the same way that we have robust cost-effectiveness estimates for the near-term effects of some things. However, there has been lots of work done prioritising between long-term causes and we do have some best guesses about the most effective things to work on.
We have a better idea of OpenAI’s long-term effects than AMF’s, just because we’ve thought more about the long-term effects of OpenAI, and it’s targeting a long-term problem.
We’re uncertain in our estimate of OpenAI’s effectiveness. This uncertainty is unfortunate but that doesn’t mean we can ignore the future people that OpenAI is trying to help. If we’re trying to help others as much as possible, we’re going to have to deal with lots of difficulties and lots of uncertainties.
Long-term effects don’t persist
Objection: What makes me think that long-term effects tend to persist in the future, rather than slowly fading out? If I drop a stone into a pond, it has a large local effect. But then the ripples spread out and eventually it’s like I never dropped the rock at all. Maybe near-term interventions are like this. This is different to saying the long-term effects ‘cancel out’ in expectation—maybe they just disappear. If that’s true, then the biggest effects of an intervention are the near-term effects.
Response: One way we can see that long-term effects seem to persist is through identity-altering actions, as described in the ‘simple cluelessness’ section above. Once my decisions affect a conception event, I am causally responsible for everything that the conceived person counterfactually does. I am causally responsible for the effects of those things and for the effects of those effects and so on. As time goes on, I will be causally responsible for more and more effects, not fewer and fewer.
(Maybe there are domains in which effects are likely to wash out rather than persist, I haven’t read anything about this though.)
What’s good in the near-term is good in the long-term
Objection: If we improve the world today, that’s likely to lead to a better world tomorrow, if the ways in which it’s better are sustainable or likely to compound. For example, if I help the poorest people now, that will put the world in a better state in 100 years time.
Response: This is basically saying that the flow through effects of near-term interventions tend to be good. As discussed earlier, I think it’s possible that they are (though this is a hard and non-obvious question). But this doesn’t mean that we should justify interventions based on their near-term effects and look for whichever interventions have the best near-term effects. To me, it implies we should look for things with the best flow though effects and justify interventions by those effects. Otherwise, we might just succumb to Goodhart’s Law.
Also, beware surprising and suspicious convergence.
Considering long-term effects leads to inconsistency
Objection: In my daily life, I don’t consider the long-term effects of my actions. If I delay someone on the street, I’m not worried about causing the next Stalin to be conceived. If I did do that, I’d never be able to do anything. It’s consistent to have a decision procedure that applies both to daily life and to improving the world.
Response: In daily life, we often have simple cluelessness because we have evidential symmetry, as described above. We have no more reason to believe that the effects of delaying someone will be good than we have to believe that they will be bad. Every way that affecting a conception event could be good, is also a way that it could be bad. However, every way that the long-term effects of a near-term intervention could be good are not the exact same ways that it could be bad. So we don’t have evidential symmetry and it’s consistent to behave differently in this different case.
Also, in daily life, we have goals that are not maximally, impartially welfarist so it makes sense to act differently.
Considering long-term effects leads to analysis paralysis
Objection: We are in triage every second of every day. Every day that we wait for better understanding of long-term effects is time that we are not helping people right now.
Response: Yes, we are in triage. We want to end factory farming, human diseases, and wild animal suffering that’s happening. We want to make sure humanity is safe from asteroids, nuclear war, and misaligned AI so that we can go on to treat all beings fairly and fill the universe with meaningful joy. We can’t do all of these things right now so we’ve decided to pick the problems where we think we can make the biggest difference. But just as triage doesn’t mean that we should necessarily prioritise the first person we see on the street, it doesn’t mean that we should necessarily prioritise beings alive right now. Triage means finding the very best opportunities for doing good and then taking them. It might be that, if we want to do the most good, we have to spend a bit more time on finding opportunities than taking them right now.
Near-term work is more aligned with elite common sense
Objection: We should have elite common sense as a prior. Long-term interventions tend to be weird, wacky, and unconventional so we should be pretty sceptical of them for outside-view reasons.
Response: The recommendation from the linked post is to believe what you think a broad coalition of trustworthy people would believe if they were trying to have accurate views and they had access to your evidence. I think there’s a way this could point to focusing on near-term effects but I can’t see what it is. My perspective is that the EA community is a broad coalition of trustworthy people who have access to my evidence and are trying to have trustworthy views. It seems like, as people spend more time in EA, they become more longtermist. So this idea seems to point to long-termism. In general, it doesn’t seem that unconventional to value the future, the unconventional bit is acting on those values. This is where EA diverges from common sense, but it does so just as much for near-term interventions as for long-term interventions (from my perspective). Ie. FHI is unconventional, but so is GiveWell.
Conclusion
It seems to me that:
Our actions have dominating long-term effects that we cannot ignore
If you care about future people, it’s best to pick your interventions based on (your best guess at) those dominating long-term effects
So, what am I missing? If you do value future people and you look to the direct effects of interventions, why is this?
- Where are you donating in 2020 and why? by (23 Nov 2020 8:47 UTC; 71 points)
- What’s your prior probability that “good things are good” (for the long-term future)? by (5 Feb 2022 18:44 UTC; 69 points)
- Strong Longtermism, Irrefutability, and Moral Progress by (26 Dec 2020 19:44 UTC; 60 points)
- Finding bugs in GiveWell’s top charities by (23 Jan 2023 16:49 UTC; 46 points)
- Solving the moral cluelessness problem with Bayesian joint probability distributions by (28 Feb 2021 9:17 UTC; 11 points)
- If someone identifies as a longtermist, should they donate to Founders Pledge’s top climate charities than to GiveWell’s top charities? by (26 Nov 2020 7:54 UTC; 10 points)
- 's comment on The case of the missing cause prioritisation research by (22 Aug 2020 19:33 UTC; 9 points)
- 's comment on What are the leading critiques of “longtermism” and related concepts by (31 May 2020 7:37 UTC; 9 points)
- 's comment on How to think about an uncertain future: lessons from other sectors & mistakes of longtermist EAs by (5 Sep 2020 22:05 UTC; 8 points)
- 's comment on A case against strong longtermism by (23 Dec 2020 18:47 UTC; 4 points)
- 's comment on MichaelStJules’s Quick takes by (27 Mar 2020 5:54 UTC; 3 points)
- 's comment on Dawn Drescher’s Quick takes by (23 Apr 2020 15:44 UTC; 3 points)
- 's comment on Finding bugs in GiveWell’s top charities by (24 Jan 2023 10:37 UTC; 2 points)
- 's comment on Thoughts on “The Case for Strong Longtermism” (Greaves & MacAskill) by (2 May 2021 18:06 UTC; 2 points)
EDIT: I think I may have been mixing risk-aversion with respect to welfare and risk-aversion with respect to the difference made by one’s intervention, as discussed in section 4.2 here. Usually, although not necessarily, a bounded utility function will be concave above some point, say 0, and convex below. Concavity implies risk-aversion and would lead you to give extra weight to avoiding particularly bad scenarios (e.g. close to or below 0) in the concave region compared to improving scenarios that are already good in the concave region. This explains why we buy insurance, and is consistent with the maxipok rule to maximize the probability of an OK outcome (which doesn’t distinguish between bad outcomes, some could be far worse than just “not okay”, as this paper discusses.)
Consistent with what I said below, a small chance of making the future really great is not as compelling as it would be if you’re risk-averse/concave above 0. However, ensuring the future is good rather than at best neutral (say extinction of all moral patients, with symmetric population ethics, or human extinction and net suffering in the wild for a long time) is more compelling than otherwise if you’re risk-averse/concave above 0.
If you think the universe is large, has extreme net utility (either negative or positive) regardless of what we do and there are orders of magnitude more moral patients we can’t affect, then it gets messier again.
Original comment follows:
I suspect the best fundamental response to Pascalian problems is to actually have your utility function bounded above and below. Whether longtermist interventions are Pascalian or not, astronomical stakes become much less compelling, and this leads to a preference for higher probabilities of making a difference that’s incompatible with risk-neutrality. I guess this is a kind of risk-aversion, although preventing extremely unlikely horrible outcomes (or making a tiny difference to their probability of occurrence) isn’t as compelling either.
A bounded utility function can’t be additive1. Lives vs headaches (or torture vs dust specks) also gives me reason to believe the value of the whole is not the sum of the value of its parts. I’d rather give up additivity (or separability or independence2) than continuity or my strong prioritarianism. See also this theorem on social welfare functions (up until axiom 5), CLR’s writing on value lexicality and its references, and Stuart Armstrong on the sadistic conclusion.
I think respecting autonomy and individuals’ preferred tradeoffs is a reason for additivity/separability/independence (see Harsanyi’s argument here and some more accessible discussions here and here, and there are other similar theorems), but not more compelling than my intuitions against it.
1. although it can be up to an increasing transformation, e.g. tan and arctan. The social welfare function arctan(∑iui) is bounded, and when there’s no uncertainty, it is just utilitarianism and produces the same rankings of choices, but with this function, you can’t in general ignore unaffected (identically distributed) individuals between choices if you have uncertainty about their utilities (and numbers in existence).
2. also called independence of unconcerned agents or independence of the utilities of the unconcerned
Also, from “The Epistemic Challenge to Longtermism” by Christian Tarsney for the Global Priorities Institute:
For what it’s worth, this doesn’t really justify the case for any particular longtermist intervention, so the case for longtermism only looks robust here if you can predictably make a net positive difference with some small but large enough probability. This probability could actually be negligible, unless you have good reason to believe otherwise.
Also, whether you think the probabilities involved are Pascalian or not, or even care, this work is super cool, and I think the talk is pretty accessible if you’re comfortable with 2nd-year undergrad probability. I definitely recommend watching/reading.
Thanks for this. (I should say I don’t completely understand it). My intuitions are much more sympathetic to additivity over prioritarianism but I see where you’re coming from and it does help to answer my question (and updates me a bit).
I wonder if you’ve seen this. I didn’t take the time to understand it fully but it looks like the kind of thing you might be interested in. (Also curious to hear whether you agree with the conclusions).
The blog post was great, thanks for sharing! I’ve come across the paper that blog post is based on, although I didn’t read through the parts on background uncertainty, which is basically the main contribution (other than arguing for stochastic dominance, which was convincing to me). I agree that stochastic dominance is the most important rationality axiom, maybe the only important one, and so whatever follows from it + background uncertainty precedes all other rationality assumptions (the work also assumes utilitarianism, which may be false). The paper is also by Christian Tarsney, and he references it in The Epistemic Challenge to Longermism and claims that the probabilities are plausibly low enough that background uncertainty dominates and we should go with the near-term intervention (from footnote 31 on pages 29-30 here):
Some other remarks on Tarsney’s stochastic dominance approach:
I think the von Neumann-Morgenstern rationality axioms (except Continuity) are actually justified based on stochastic dominance and certain (usually unstated) assumptions about how to treat certain sequences of decisions, using money pumps/Dutch books. The point is to trick you into choosing an option that’s stochastically dominated by another. If we accept these assumptions + Continuity, then we should have a bounded vNM utility function. Tarsney’s results don’t conflict with this, but if you want to avoid Pascal’s wager (or similar with tiny probabilities of infinite payoffs according to utilitarianism) and still satisfy the assumptions, then you need to accept Continuity, and your vNM utility function must be bounded.
It also gives up the kind of additivity over uncertainty I described in point 1 in my comment. How good an action is can depend on your beliefs about parts of the universe that are totally unaffected by your action, even outside the observable universe. Tarsney defends this in section 7.
The value in the entire universe (not just the observable part) is undefined or infinite (positive or negative, but can’t be affected) with high probability, since the universe is infinite/unbounded spatially with high probability, so if you have symmetric views, there’s both infinite positive value and infinite negative value, and the order in which you sum matters. Stochastic dominance either breaks down or forces us to ignore this part of the probability space if our impact is finite at most. Additivity with uncertainty as I described in point 1 allows us to ignore parts of the universe we can’t affect.
Only we can help those who are suffering now. Future people will be able to influence those who will live in the future. So I don’t think that the hotel analogy is quite right. We press a button knowing what it will do to the occupant in the room #327 right now. What happens to other occupants will depend on many other people pressing buttons afterwards, and these other people will be in better positions to optimize for other rooms. Which button we press influences which buttons/options they will have, but it doesn’t directly cause occupants bliss or torture in a way that we can predict. In this situation it’s unclear whether it’s better to optimize for room #327, or to make some fancy theory where we try to predict what buttons/options other people will have and which buttons they will decide to press.
Thanks for the answer Saulius, and I agree the hotel analogy is pretty different to the reality! So do you think the long-term effects don’t dominate? Or we can’t say what they are because they depend on other people’s unpredictable behaviour in a way that near-term things don’t?
And I think you’re also saying that, at any given time, we have a special opportunity to influence that time. Is that because we have more evidence about present effects or because there’s something special about direct rather than indirect effects? I’m confused because it seems like while we do have a special opportunity to influence the present because we’re here now, we also have a special opportunity to influence the future too because we’re here now. Eg. by doing anything that has positive compounding effects, or avoids lock-in of a bad state.
I just also want to say that in general, I really appreciate you engaging in this discussion and writing this post, especially in such a clear and well-structured way. I think that criticising others’ views takes courage but can be very valuable.
I’ve spent a lot of time discussing these questions and I still don’t have a strong opinion. I like the basic idea behind OpenPhil’s worldview diversification approach. They “allocate X% of capital to a bucket that aims to maximize impact from a “long-termist” perspective, and Y% of capital to a bucket that aims to maximize impact from a “near-termist” perspective”. X and Y are determined by how much credence they place on each worldview. If I was responsible for giving away that much money, I’d probably do the same. As an individual, I had to specialize and I found it easier to get a job on short-termist stuff so that’s what I’m working on.
I also like that basic idea, though there’s also another part of me that feels deeply unsatisfied with it.
In any case, some other ideas that seem relevant are moral parliaments and moral uncertainty more generally.
Yes, I think this is just a reformulation of the old question, can we make more impact when focusing on short-term future or long-term future. And I think there is no easy answer to it.
Do you think any of these things have positive compounding effects or avoid lock-in:
-Investing to donate later,
-Narrow x-risk reduction,
-Building the EA community?
Yes, all of these. But e.g. donation opportunities later could get worse to the point where this is outweighed. Also, with all of these, you empower future people (or future you) to do good rather than doing good directly but it’s unclear if they will use that power in a good way.
Two sources which I think discuss at least sort-of relevant things quite well are:
The timing of labour aimed at reducing existential risk—Toby Ord
80k interview with Phil Trammell
(You may already be aware of these.)
I think a caricature/extreme version of a related view would be “Progress is practically guaranteed to continue to the point where everything eventually becomes as good as it could be. Therefore, there’s no need to try to improve the long-run future, and what we should do is just make things go better until that point, or help us get to that point faster.”
I don’t know if anyone confidently holds that a view quite that extreme. But I think it’s relatively common to think that there’s a decent chance that something like that is true, and that that’s probably one of the common reasons for people not prioritising “longtermist interventions”.
Personally, I think that believing there’s a decent chance that something like that is true probably makes sense. However, I currently believe it’s sufficiently likely that we’re at something like a hinge of history, where that march of progress could be foiled, that longtermist work makes sense. And I also believe we can reach a similar conclusion from the idea that, even if we avoid x-risks and bad lock-in, we may not be guaranteed to reach an optimal point “by default” (e.g., maybe moral circles won’t expand far enough, or we’ll get stuck in some bad equilibria), so longtermist “trajectory change” work could be valuable.
(My point here is more to try to highlight some views than to argue for or against them.)
Is the idea that most of the opportunities to do good will be soon (say in the next 100-200 years)? Eg. because we expect less poverty, and factory farms etc. Or because the AI is gonna come and make us all happy, so we should just make the bit before that good?
Distinct from that seems ‘make us get to that point faster’ (I’m imagining this could mean things like increasing growth/creating friendly AI/spreading good values) - that seems very much like looking to long-term effects.
I think there’s a decent number of people who give a decent amount of credence to either or both of those possibilities. (I guess I count myself among such people, but also feel wary about having high confidence in those claims, and I see it as very plausible progress will be disrupted in various ways.) People may also believe the first thing because the believe the second thing; e.g., we’ll develop very good AI—doesn’t necessarily have to be agenty or superintelligent—and that will allow us to either suddenly or gradually-but-quickly eliminate poverty, develop clean meat, etc.
One way speeding things up is distinct is that it also helps with allowing us to ultimately access more resources (the astronomical waste type argument). But it mostly doesn’t seem very distinct to me from the other points. Basically, you might think we’ll ultimately reach a fairly optimal state, so speeding things up won’t change that, but it’ll change how much suffering/joy there is before we get to that state. This sort of idea is expressed in the graph on the left here.
So I feel like maybe I’m not understanding that part of your comment?
(I should hopefully be publishing a post soon disentangling things like existential risk reduction, speed-ups, and other “trajectory change” efforts. I’ll say it better there, and give pretty pictures of my own :D)
Ah yeah that makes sense. I think they seemed distinct to me because one seems like ‘buy some QALYS now before the singularity’ and the other seems like ‘make the singularity happen sooner’ (obviously these are big caricatures). And the second one seems like it has a lot more value than the first if you can do it (of course I’m not saying you can). But yeah they are the same in that they are adding value before a set time. I can imagine that post being really useful to send to people I talk to—looking forward to reading it.
If it’s extremely difficult to figure out the direct effects of near-term interventions, then maybe it’s proportionally harder to figure out long term effects—even to the point of complex cluelessness becoming de facto simple cluelessness.
Some people argue from a “skeptical prior”: simply put, most efforts to do good fail. The international development community certainly seems like a “broad coalition of trustworthy people”, but their best guesses are almost useless without hard evidence.
If you’re GiveWell-level pessimistic about charities having their intended impact even with real time monitoring and evaluation of measurable impacts, you might be utterly and simply clueless about all long term effects. In that case, long term EV is symmetrical and short term effects dominate.
Provably successful near-term work could drive the growth of the EA movement, benefitting the long term. I’d guess that more people join EA because of GiveWell and AMF than because of AI Safety and biorisk. That’s because (a) near-term work is more popular in the mainstream, and (b) near-term work can better prove success. More obvious successes will probably drive more EA growth. On the other hand, if EA makes a big bet on AI Safety and 30 years from now we’re no closer to AGI or seeing the effects of AI risks, the EA movement could sputter. It’s hard to imagine demonstrably failing like that in near-term work. Maybe the best gift we can give the future isn’t direct work on longtermism, but is rather enabling the EA movement of the future.
I’m not actually sure I buy this argument. If we’re at the Hinge of History and we have more leverage over the expected value of the future than anyone in the future will, maybe some longtermist direct work now is more important than enabling more longtermist direct work in the future. Also, maybe EA’s best sales pitch is that we don’t do sales pitches, we follow the evidence even to less popular conclusions like longtermism.
I think the natural response to your Hinge of History response is that perhaps we’re not currently at the Hinge of History, but we may be in ~10-20 years, and so the key is about building the EA movement so that it is best prepared for the actual (theoretical/hypothetical) Hinge of History period.
One concern with complex cluelessness is that you actually don’t know the magnitudes of these causal effects. If you tell me X causes an increase in Y without justifying an effect size, I’ll be skeptical that the effect size is large, and I can be arbitrarily skeptical. Also, longer causal chains without feedback are much less robust: it’s hard to know the net effect of X on Y, since it’s more likely that there are important alternative causal paths you haven’t accounted for. Of course, there’s still generalization error with feedback, and I don’t think this is a fundamentally different kind of error, but I’m much less skeptical with feedback. Also, sometimes generalization error can be estimated,[1][2] but then there’s generalization error on this generalization error...
I have some thoughts about this on my shortform here. To summarize, I’m really skeptical of causal effects.
I would still say those aren’t persistent causal effects ex ante or the kind that matter, if your distributions over future identities are the same as in the simple cluelessness case (by symmetry). I think you need an example where you can justify that the outcome distributions are significantly different. I actually haven’t been convinced that this is the case for any longtermist intervention, but I’m looking more into it now.
So, as above, I rely primarily on causal effects I’m confident in. I’m skeptical of net long-term effects of interventions in any particular direction.
Agreed that, at least from a utilitarian perspective, identity effects aren’t what matter and feel pretty symmetrical, and that they’re therefore not the right way to illustrate complex cluelessness. But when you say
—maybe I’m misunderstanding you, but I believe the proposition being defended here is that the distribution of long-term welfare outcomes from a short-termist intervention differs substantially from the status quo distribution of long-term welfare outcomes (and that this distribution-difference is much larger than the intervention’s direct benefits). Do you mean that you’re not convinced that this is the case for any short-termist intervention?
Even though we don’t know the magnitudes of today’s interventions’ long-term effects, I do think we can sometimes confidently say that the distribution-difference is larger than the direct effect. For instance, the UN’s 95% confidence interval is that the population of Africa will multiply by about 3x to 5x by 2100 (here, p.7). One might think their confidence interval should be wider, but I don’t see why the range would be upwards-biased in particular. Assuming that fertility in saved children isn’t dramatically lower than population fertility, this strikes me as a strong reason to think that the indirect welfare effects of saving a young person’s life in Africa today—indeed, even a majority of the effects on total human welfare before 2100—will be larger than the direct welfare effect.
Saving lives might lower fertility somewhat, thus offsetting this effect, But the (tentative) conclusion of what I believe is the only in-depth investigation on this is that there are some regions in which this offsetting is negligible. And note that if those UN projections are any guide, the fertility-lowering effect would have to be not just non-negligible but very close to complete for the direct welfare effects to outweigh the indirect.
Does that seem wrong to you?
No, that seems plausible, although I’d have to look into how long the population effects go. The point isn’t about direct vs indirect effects (all effects are indirect, in my view), but net effects we have estimates of magnitude for. I don’t consider those effects to be “long-term” in the way longtermists use the word. The expected value on the long term isn’t obvious at all, since there are too many different considerations to weigh against one another, many we’re unaware of, and no good way to weigh them.
Note: I retracted my previous reply.
To be specific (and revising my claim somewhat), I’m not convinced of any net expected longterm effect in any particular direction on my social welfare function/utility function. I think there are many considerations that can go in either direction, the weight we give them is basically arbitrary, and I usually don’t have good reason to believe their effects persist very long or are that important, anyway.
I am arguing from ignorance here, but I don’t yet have enough reason to believe the expected effect is good or bad. Unless I expect to be able to weigh opposing considerations against one another in a way that feels robust and satisfactory to me and be confident that I’m not missing crucial considerations, I’m inclined to not account for them until I can (but also try to learn more about them in hope of having more robust predictions). A sensitivity analysis might help, too, but only so much. The two studies you cite are worth looking into, but there are also effects of different population sizes that you need to weigh. How do you weigh them against each other?
What’s the expected value (on net) of the indirect effects to you? Is its absolute value much greater than the direct effects’ expected value? How robust do you think the sign of the expected value of the indirect effects is to your subjective weighting of different considerations and missed considerations?
Also, what do you think the expected change in population size is from saving one life through AMF?
Hold on—now it seems like you might be talking past the OP on the issue of complex cluelessness. I 1000% agree that changing population size has many effects beyond those I listed, and that we can’t weigh them; but that’s the whole problem!
The claim is that CC arises when (a) there are both predictably positive and predictably negative indirect effects of (say) saving lives which are larger in magnitude than the direct effects, and (b) you can’t weigh them all against each other so as to arise at an all-things-considered judgment of the sign of the value of the intervention.
A common response to the phenomenon of CC is to say, “I know that the direct effects are good, and I struggle to weigh all of the indirect effects, so the latter are zero for me in expectation, and the intervention is appealing”. But (unless there’s a strong counterargument to Hilary’s observation about this in “Cluelessness” which I’m unaware of), this response is invalid. We know this because if this response were valid, we could by identical reasoning pick out any category of effect whose effects we can estimate—the effect on farmed chicken welfare next year from saving a chicken-eater’s life, say—and say “I know that the next-year-chicken effects are bad, and I struggle to weigh all of the non-next-year-chicken effects, so the latter are zero for me in expectation, and the intervention is unappealing”.
The above reasoning doesn’t invalidate that kind of response to simple cluelessness, because there the indirect effects have a feature—symmetry—which breaks when you cut up the space of consequences differently. But this means that, unless one can demonstrate that the distribution of non-direct effects has a sort of evidential symmetry that the distribution of non-next-year-chicken effects does not, one is not yet in a position to put a sign to the value of saving a life.
So, the response to
is that, given an inability to weigh all the effects, and an absence of evidential symmetry, I simply don’t have an expected value (or even a sign) of the indirect effects, or the total effects, of saving a life.
Does that clarify things at all, or am I the one doing the talking-past?
Sorry, I misunderstood your comment on my first reading, so I retracted my first reply.
No worries, sorry if I didn’t write it as clearly as I could have!
BTW, I’ve had this conversation enough times now that last summer I wrote down my thoughts on cluelessness in a document that I’ve been told is pretty accessible—this is the doc I link to from the words “don’t have an expected value”. I know it can be annoying just to be pointed off the page, but just letting you know in case you find it helpful or interesting.
I’m not sure if what I’m defending is quite the same as what’s in your example. It’s not really about direct or indirect effects or how to group effects to try to cancel them; it’s just skepticism about effects.
I’ll exclude whichever I don’t have a good effect size estimate on my social welfare function for (possibly multiple), since I’ll assume the expected effect size is small. If I have effect sizes for both, then I can just estimate the net effect. As a first approximation, I’d just add the two effects. If I have reason to believe they should interact in certain ways and I can model this, I might.
If you’re saying I know the two opposite sign indirect effects are larger in magnitude than the direct ones, it sounds like I have estimates I can just sum (as a first approximation). Is the point that I’m confident they’re larger in magnitude, but still not confident enough to estimate their expected magnitudes more precisely?
Maybe I have a good idea of the impacts over each possible future, but I’m very uncertain about the distribution of possible futures. I could be confident about the sign of the effect of population growth when comparing pairs of counterfactuals, one with the child saved, and the other not, but I’m not confident enough to form distributions over the two sets of counterfactuals to be able to determine the sign of the expected value.
I think I’m basically treating each effect without an estimate attached independently like simple cluelessness. I’m not looking at a group of positive and negative effects and assuming they cancel; I’m doubting the signs of the effects that don’t come with estimates. If I have a plausible argument that doing X affects Y and Y affects Z, which I value directly and the effect should be good, but I don’t have an estimate for the effect through this causal path, I’m not actually convinced that the effect through this path isn’t bad.
Now, I’m not relying on a nice symmetry argument to justify this treatment like simple cluelessness, but I’m also not cutting up the space of consequences and ignoring subsets; I’m just ignoring each effect I’m skeptical of.
This does push the problem to which effects I should try to estimate, though.
Yes, exactly—that’s the point of the African population growth example.
I don’t understand this paragraph. Could you clarify?
I don’t think I understand this either:
Say you have a plausible argument that pushing a switch (doing X) pulls some number n > 0 of strings (so Y := #strings_pulled goes from 0 to n), each of which releases some food to m > 0 hungry lab mice (so Z := #fed_mice goes from 0 to nm), and you know that X and Y have no other consequences. You know that n, m > 0 but don’t have estimates for them. At face value you seem to be saying you’re not convinced that the effect of pushing the switch isn’t bad, but that can’t be right!
Population growth will be net good or bad depending on my credences about what the future would have looked like, but these credences are not robust. E.g. I might think it’s bad in cases like X and good in cases like notX and have conditional expectations for both, but I’m basically just guessing the probability of X, and which is better depends on the probability of X (under each action).
So the assumption here is that I think the effect is nonnegative with probability 1. I don’t think mere plausibility arguments or considerations give me that kind of credence. As a specific example, is population growth actually bad for climate change? The argument is “More people, more consumption, more emissions”, but with no numbers attached. In this case, I think there’s some probability that population growth is good for climate change, and without estimates for the argument, I’d assume the amount of climate change would be identically distributed with and without population growth. Of course, in this case, I think we have enough data and models to actually estimate some of the effects.
Even with estimates, I still think there’s a chance population growth is good for climate change, although my expected value would be that it’s bad. It could depend on what kind of people the extra people are like, and what kinds of effects they have on society.
Suppose for simplicity that we can split the effects of saving a life into
1) benefits accruing to the beneficiary;
2) benefits accruing to future generations up to 2100, through increased size (following from (1)); and
3) further effects (following from (2)).
It seems like you’re saying that there’s some proposition X such that (3) is overall good if X and bad if not-X, where we can only guess at the probability of X; and that in this circumstance we can say that the overall effect of (2 & 3) is ~zero in expectation.
If that’s right, what I’m struggling to see is why we can’t likewise say that there’s some proposition Y such that (2 & 3) is overall good if Y and bad if not-Y, where we can only guess at the probability of Y, and that the overall effect of (1 & 2 & 3) is therefore ~zero in expectation.
I wasn’t saying we should cancel them this way; I’m just trying to understand exactly what the CC problem is here.
What I have been proposing is that I’m independently skeptical of each causal effect that doesn’t come with effect size estimates (and can’t, especially), as in my other comments, and Saulius’ here. If you give me a causal model, and claim A has a certain effect on B, without justifying rough effect sizes, I am by default skeptical of that claim and treat that like simple cluelessness: B conditional on changing A is identically distributed to B. You have not yet justified a systematic effect of A on B.
However, I’m thinking that I could be pretty confident about effect sizes conditional on X and notX, but have little idea about the probability of X. In this case, I shouldn’t just apply the same skepticism, and I’m stuck trying to figure out the probability of X, which would allow me to weigh the different effects against each other, but I don’t know how to do it. Is this an example of CC?
What I’m saying is, “Michael: you’ve given me a causal model, and claimed A (saving lives) has a positive effect on B (total moral value in the universe, given all the indirect effects), without justifying a rough effect size. You just justified a rough effect size on C (value to direct beneficiaries), but that’s not ultimately what matters. By default I think A has no systematic effect on B, and you have not yet justified one.”
Yes, you have CC in that circumstance if you don’t have evidential symmetry with respect to X.
The value to the universe is the sum of values to possible beneficiaries, including the direct ones C, so there is a direct and known causal effect of C on B.u1 has a causal effect on ∑iui, under any reasonable definition of causal effect, and it’s the obvious one: any change in u1 directly causes an equal change in the sum, without affecting the other terms. The value in my life (or some moment of it) u1 doesn’t affect yours u2, although my life itself or your judgment about my u1 might affect your life and your u2. Similarly, any subset of the ui (including C) has a causal effect on the sum.
If you think A has no effect on B (in expectation), this is a claim that the effects through C are exactly negated by other effects from A (in expectation), but this is the kind of causal claim that I’ve been saying I’m skeptical of, since it doesn’t come with a (justified) effect size estimate (or even an plausible argument for how this happens, in this case).
This is pretty different from the skepticism I have about long-term effects: in this case, people are claiming that A affects a particular set of beneficiaries C where C is in the future, but they haven’t justified an effect size of A on C in the first place; many things could happen before C, completely drowning out the effect. Since I’m not convinced C is affected in any particular way, I’m not convinced B is either, through this proposed causal chain.
With short term effects, when there’s good feedback, I actually have proxy observations that tell me that in fact A affects C in certain ways (although there are still generalization error and the reference class problem to worry about).
At the risk of repetition, I’d say that by the same reasoning, we could likewise add in our best estimates of saving a life on (just, say) total human welfare up to 2100.
Your response here was that “[p]opulation growth will be net good or bad depending on my credences about what the future would have looked like, but these credences are not robust”. But as with the first beneficiary, we can separate the direct welfare impact of population growth from all its other effects and observe that the former is a part of “sum u_i”, no?
Of course, estimates of shorter-term effects are usually more reliable than those of longer-term effects, for all sorts of reasons; but since we’re not arguing over whether saving lives in certain regions can be expected to increase population size up to 2100, that doesn’t seem to me like the point of dispute in this case.
I’m not sure where we’re failing to communicate exactly, but I’m a little worried that this is clogging the comments section! Let me know if you want to really try to get to the bottom of this sometime, in some other context.
I’m not trying to solve all complex cluelessness cases with my argument. I think population growth is plausibly a case with complex cluelessness, but this depends on your views.
If I were a total utilitarian with symmetric population ethics, and didn’t care much about nonhuman animals (neither of which is actually true for me), then I’d guess the negative externalities of a larger population would be strongly dominated by the benefits of a larger population, mostly just the direct benefits of the welfare of the extra people. I don’t think the effects of climate change are that important here, and I’m not aware of other important negative externalities. So for people with such views, it’s actually just not a case of complex cluelessness at all. The expectation that more people than just the one you saved will live probably increases the cost-effectiveness to someone with such views.
Similarly, I think Brian Tomasik has supported the Humane Slaughter Association basically because he doesn’t think the effects on animal population sizes and wild animals generally are significant compared to the benefits. It does good with little risk of harm.
So, compared to doing nothing (or some specific default action), some actions do look robustly good in expectation. Compared to some other options, there will be complex cluelessness, but I’m happy to choose something that looks best in expectation compared to doing nothing. I suppose this might privilege a specific default action to compare to in a nonconsequentialist way, although maybe there’s a way that gives similar recommendations without such privileging (I’m only thinking about this now):
You could model this as a partial order with A strictly dominating B if the expected value of A is robustly greater than the expected value of B. At least, you should never choose dominated actions. You could also require that the action you choose dominates at least one action when there is any domination in the set of actions you’re considering, and maybe this would handle a lot of complex cluelessness, if actions are decomposed enough into pretty atomic actions. For example, with complex cluelessness about saving lives compared to nothing, saving a life and punching myself in the face is dominated by saving a life and not punching myself in the face, but I can treat saving a life or not and at a separate time punching myself in the face or not as two separate decisions.
(Focusing on a subtopic of yours, rather than engaging with the entire argument.)
I’m not so sure “all actions we take have huge effects on the future.” It seems like a pretty interesting empirical question. I don’t find this analogy supremely convincing; it seems that life contains both “absorbers” and “amplifiers” of randomness, and I’m not sure which are more common.
In your example, I stop to chat with my friend vs. not doing so. But then I just go to my job, where I’m not meeting any new people. Maybe I always just slack off until my 9:30am meeting, so it doesn’t matter whether I arrive at 9am or at 9:10am after stopping to chat. I just read the Internet for ten more minutes. It looks like there’s an “absorber” here.
Re: conception events — I’ve noticed that discussion of this topic tends to use conception as a stock example of an amplifier. (I’m thinking of Tyler Cowen’s Stubborn Attachments.) Notably, it’s an empirical fact that conception works that way (e.g. with many sperm, all with different genomes, competing to fertilize the same egg). If conception did not work that way, would we lower our belief in “all actions we take have huge effects on the future” ? What sort of evidence would cause us to lower our beliefs in that?
Sure, but what about the counterfactual? How much does it matter to the wider world what this person’s traits are like? You want JFK to be patient and levelheaded, so he can handle the Cuban Missile Crisis. JFK’s traits seem to matter. But most people aren’t JFK.
You might also have “absorbers,” in the form of selection effects, operating even in the JFK case. If we’ve set up a great political system such that the only people who can become President are patient and levelheaded, it matters not at all whether JFK in particular has those traits.
Looking at history with my layman’s eyes, it seems like JFK was groomed to be president by virtue of his birth, so it did actually matter what he was like. At the extreme of this, kings seem pretty high-variance. So affecting the conception of a king matters. But now what we’re doing looks more like ordinary cause prioritization.
What do you think absorbers might be in cases of complex cluelessness? I see that delaying someone on the street might just cause them to spend 30 seconds less procrastinating, but how might this work for distributing bednets, or increasing economic growth?
Maybe there’s an line of argument around nothing being counterfactual in the long-term - because every time you solve a problem someone else was going to solve it eventually. Eg. if you didn’t increase growth in some region, someone else would have 50 years later. And now you did it they won’t. But this just sounds like a weirdly stable system and I guess this isn’t what you have in mind
I suppose an example would be that increasing economic growth in a country doesn’t matter if the country later gets blown up or something.
I found this a really interesting way of framing and phrasing this sort of thing. Thanks!
I think some other things that are relevant to this subtopic include:
https://en.wikipedia.org/wiki/Historical_determinism
https://www.sentienceinstitute.org/blog/how-tractable-is-changing-the-course-of-history
https://forum.effectivealtruism.org/posts/n8nXqqgbwp58wuo6a/how-fragile-was-history
Sweet links, thanks!
I also found this to be a great framing of absorbers and hadn’t really got this before. It’s an argument against ‘all actions we take have huge effects on the future’, and I’m not sure how to weigh them up against each other empirically. Like how would I know if the world was more absorber-y or more sensitive to small changes?
I think conception events are just one example and there a bunch of other examples of this, the general idea being that the world has systems which are complex, hard to predict and very sensitive to initial conditions. Eg. the weather and climate system (a butterfly flapping its wings in China causing hurricane in Texas). But these are cases of simple cluelessness where we have evidential symmetry.
My claim is that we are faced with complex cluelessness, where there are some kind of systematic effects going on. To apply this to conception events—imagine we changed conception events so that girls were much more likely to be conceived than boys (say because in the near-term that had some good effects eg. say women tended to be happier at the time). My intuition here is that there could be long-term effects of indeterminate sign (eg. from increased/decreased population growth) which might dominate the near-term effects. Does that match your intuition?
I’m not sure; that’s a pretty interesting question.
Here’s a tentative idea: using the evolution of brains, we can conclude that whatever sensitivity the world has to small changes, it can’t show up *too* quickly. You could imagine a totally chaotic world, where the whole state at time t+(1 second) is radically different depending on minute variations in the state at time t. Building models of such a world that were useful on 1 second timescales would be impossible. But brains are devices for modelling the world that are useful on 1 second timescales. Brains evolved; hence they conferred some evolutionary advantage. Hence we don’t live in this totally chaotic world; the world must be less chaotic than that.
It seems like this argument gets less strong the longer your timescales are, as our brains perhaps faced less evolutionary pressure to be good at prediction on timescales of like 1 year, and still less to be good at prediction on timescales of 100 years. But I’m not sure; I’d like to think about this more.
Hey, glad this was helpful! : )
Yes, that matches my intuition. This action creates a sweeping change a really complex system; I would be surprised if there were no unexpected effects.
But I don’t see why we should believe all actions are like this. I’m raising the “long-term effects don’t persist” objection, arguing that it seems true of *some* actions.
Sometimes I feel like this:
You could work on long term cause. We’re not sure if it will be positive or negative, but hopefully it will be positive! The actual value will be between −10,000 and +10,000.
You could work on short term cause. We’re not sure if it will be positive or negative, but hopefully it will be positive! The actual value will be 10 + (somewhere between −10,000 and +10,000).
On the one hand, clearly long term considerations dominate. On the other hand, the short term considerations seem to be the only thing we know anything about!
Nice, that’s well put. Do you think we can get any idea of longterm effects eg. (somewhere between −10,000 and +10,000, but tending towards the higher/lower end)?
I’m not sure! There might be some situations where you can make an educated guess, but it seems really hard to say
Your long-term effects don’t persist response:
I don’t think you made a convincing point here, I don’t see how simple cluelessness is relevant here. Since we have no chance to predict how these identity-altering actions will pan out, we can’t take them into account when making decisions. We only account for what we can predict.
For some short-term interventions, as time goes on, there are fewer and fewer consequences that we can predict. For example, let’s take my decision to eat a vegan meal today rather than meat to help animals (without anyone else noticing it). I can’t predict any effect it will have in 1000 years so I don’t worry about it.
A more complicated example can be something like clean meat research. Let’s imagine that I believe that no matter what we do, in 1000 years there will be no animal farming. Either we will be making clean meat, not eating meat at all, or humanity will be extinct altogether. So me donating to clean meat research is like jumping on a sinking animal farming ship to make it sink faster. The end result will be the same no matter what I do. Sure, I can make some fancy theory about how donating to this research will influence the views of beings alive in 1000 years. But I will have so little confidence in this theory that it won’t dominate my expected value calculation.
Expected value of an action = predicted short-term effect ✕ probability that I predicted short-term effect correctly + predicted long-term effect ✕ probability that I predicted long-term effect correctly. No matter what research I do, the probability that I predicted long-term effect correctly can be so low that it won’t matter much and won’t dominate the estimation.
Note that this is only an example of views I’d find reasonable, and not necessarily my actual views about clean meat.
Nice, thanks for the explanation of your reasoning.
The example I gave there was the same as for simple cluelessness, but it needn’t be. (I like this example because it shows that, even for simple cluelessness, there isn’t wash out.) For example, if we imagine some version of complex cluelessness we can see that ripples on a pond objection doesn’t seem to work. Eg. increased economic growth —> increased carbon emissions —> increased climate change —> migration problems and resources struggles —> great power conflict etc. As time goes on, the world where the extra economic growth happened will look more and more different from the world where it didn’t happen. Does that seem true?
I agree that we don’t know how to predict a bunch of these long-term effects, and this only gets worse as the timescales get longer. But why does that mean we can ignore them? Aren’t we interested in doing the things with the best effects (all the effects)? Does it matter whether we can predict the effects at the moment? Like does GiveWell doing an analysis of AMF mean that there are now better effects from donating to AMF? That doesn’t seem right to me. It does seem more reasonable to donate after the analysis (more subjectively choice-worthy or something like that). But the effects aren’t better, right? Similarly, if there are unpredictable long-term effects, why does it matter (morally*) whether that the effects are unpredictable?
With regards to that EV calculation, I think that might be assuming you have precise credences. If we’re uncertain in our EV estimates, don’t we need to use imprecise credences? Then we’d have a bunch of different term like
EV under model n*credence in model n
*or under whatever value system is motivating you/me eg subjective preferences
I think we should only ignore long-term effects that we can’t reasonably hope to predict (e.g. how conception events are changed). With clean meat research, I see no strong argument why it would affect the views of people 1000 years into the future, at least not in a negative way. So it’s not a dominant consideration for me. And I don’t put much effort into researching these very speculative considerations because I don’t think that I will come to meaningfully strong conclusions even after years of research. I would spend a day or two thinking about long-term effects before donating though.
With AMF or increased economic growth, I do see arguments about how they could negatively affect the future, so I would strongly consider them. When I used to follow GiveWell many years ago, almost all of their reasoning on the economic growth topic was here. I felt that it was not enough and I wanted them to do much more research on it. I don’t know if they did.
I do think effects matter morally no matter if they are predictable or not. It’s just if all the arguments for the impact on the long-term future are very uncertain and I don’t see how stronger arguments could be made, long-term effects don’t dominate my estimations.
There were parts of your comment that I didn’t manage to understand so I apologize if we are talking past each other.
IIRC, one prominent short-termist EA told me that when they have so little belief in speculative vague arguments that most questions with no clear answers defaults to them to 50-50 split. E.g. they would probably say that they have a 50% credence that clean meat research will have a positive value in 1000 years and a 50% that it will have a negative value, and EV is zero. You can see why they would focus on the short-term stuff. I just thought that this extreme view could be helpful to remembering how short-termist perspective might look like.
That’s basically what I’m describing in my comments here, too. If I can’t estimate the effects of an action on X, I’m thinking of just treating the two distributions for X as identical (although the random variables are different). This is similar to simple cluelessness.
Yeah that sounds like simple cluelessness. I still don’t get this point (whereas I like other points you’ve made). Why would we think the distributions are identical or the probabilities are exactly 50% when we don’t have evidential symmetry?
I see why you would not be sure of the long-term effects (not have an EV estimate), but not why you would have an estimate of exactly zero. And if you’re not sure, I think it makes sense to try to get more sure. But I think you guys think this is harder than I do (another useful answer you’ve given).
Basically, I don’t have enough reason to believe we don’t have evidential symmetry, because the proposed systematic causal effects (even if you separate different kinds of effects or considerations) aren’t quantified, even roughly, with enough justification. You have no reason to believe that the probability that the outcome from action A will be better than x (a deterministic outcome or value) 1000 years from now with a probability p>0 higher than the outcome from action B, for any probability difference p>0 or any x:
(Compare to the definition of stochastic dominance. You can replace the strict >‘s with ≥’s, except for p>0.)
So, I assume P[A1000>x]=P[B1000>x] for all x.
EDIT: Also, you can also compare the distributions of outcomes of actions A and B 1000 years from now, and again, I don’t have reason to believe pA1000(x) and pB1000(x) differ by any p>0, for any x, or P[A1000∈X]−P[B1000∈X]>p>0 for any set of outcomes X for any p>0.
Also, even if my EV is 0 and I’m treating it like simple cluelessness, can it not still make sense to try to learn more? Is the value of information under simple cluelessness necessarily 0?
It’s becoming increasingly apparent to me how strong an objection to longtermist interventions this comment is. I’d be very keen to see more engagement with this model.
My own current take: I hold out some hope that our ability to forecast long-term effects, at least under some contingencies within our lifetimes, will be not-terrible enough. And I’m more sympathetic to straightforward EV maximization than you are. But the probability of systematically having a positive long-term impact by choosing any given A over B seems much smaller than longtermists act as if is the case — in particular, it does seem to be in Pascal’s mugging territory.
This is a claim that researching these effects will cause us to know enough to change my mind about what to do, but I’m actually by default skeptical of this. The reason for my skepticism is in your next sentence:
Of course, sometimes researching effects looks better to me ahead of time, if I think the evidence I’ll get out of it will be of sufficient quality. I’m also okay with research that’s less rigorous than RCTs, but I do still have standards for rigour that long-term-focused interventions don’t meet.
In the case of animal welfare specifically, my expectation is that cultured and plant-based animal products will mostly replace real animal products soon-ish (in the next 200 years), but I’m not confident in any specific attempts to speed this up, while I am confident about the value of, say, corporate campaigns. I’m separately skeptical of the importance of the effects on things like momentum and complacency; I am not assuming they cancel.
In the case of development, there are economic growth projections, too, and I might not be confident about specific attempts to speed this up.
In both cases, there might be limited value in further studying the issues, because it will take time to get convincing data that could change your mind about what to do, the probability that this even happens is low if the evidence won’t be rigorous enough, and by the time you would have convincing data, all the low-hanging fruit could be gone.
Of course, I’m not actually confident about this. I’d want to review what evidence we have now first, try to come up with causal models for their effects, and think more about what it would take to change my mind if I’m still skeptical of the value of these other interventions. But after doing so, I might remain skeptical of the value of further research and these other interventions.
Ah ok. Can you say a bit more about why long-term-focused interventions don’t meet your standards for rigour? I guess you take speculation about long-term effects as Bayesian evidence, but only as extremely weak evidence compared to evidence about near-term effects. Is that right?
I think I basically completely discount speculation about long-term effects unless it comes with an effect size estimate justified by observation, and I haven’t seen any (although they might still be out there).
On the other hand, we can actually observe short-term effects (from similar interventions or the same intervention in a different context).
I think I’m particularly skeptical of the benefits of technical research for longtermist interventions, e.g. technical AI safety research, since there’s little precedent or feedback. For example, “How much closer does this paper bring us to solving AI safety?” My impression is that it’s basically just speculation that the research does anything useful at all in expectation. I’ve been meaning to get through this, though, and then there’s a separate question about the quality of research, especially research that doesn’t get published in journals (although some of it is).
There are also the reference class problem and issues in generalizability, and we can’t know how bad they are for longtermist work, since we don’t have good feedback.
My guess is that people who support AMF, SCI, or GiveDirectly don’t think the negative long-term effects are significant compared to the benefits, compared to “doing nothing”. These do more good than harm under a longtermist framework. Compared to “doing nothing”, they might generally just be skeptical of the causal effects of any interventions primarily targeting growth and all other so far proposed longtermist interventions (the causal evidence is much weaker) or believe these aren’t robustly good because of complex cluelessness.
I focus on animal welfare, and it’s basically the same argument for me.
If I think doing X is robustly better than doing nothing in expectation, and no other action is robustly better than doing X in expectation, then I’m happy to do X. See also my comment here.
This is the maximality rule from Maximal Cluelessness by Andreas Mogensen. Publication, GPI page (pdf), EA Forum post.
Nice, thanks
In my reading, the 80,000 Hours article in the link does not fully support this claim. In the section “Can we actually influence the future,” it identifies four ways actions today can influence the long-term future. But it doesn’t provide a solid case about why most interventions would influence the long-term future, rather than having their effects dissipate over time.
I just want to mention one more post that has some relevance here: Why I’m skeptical about unproven causes (and you should be too)
I appreciate the clarity and structure of this post, and I essentially agree with its conclusions (e.g., I’ve switched into a longtermism-aligned career). On the other hand, I think some of the arguments given don’t necessarily support the conclusions, and that there are some other “objections” some people hold which you haven’t note (some of which other commenters have already noted). I’ll put separate points in separate comments.
To me, that’s perhaps the most important pair of sentences in this post. I think this is a key point that people often miss. I believe Rob Wiblin discusses similar matters in the second interview here.
For the same reason, I also agree with the following sentence:
However, it may be worth noting that supporting longtermist interventions like AI safety is probably also more likely than AMF to have substantial negative effects, even if it’s still better in expectation.
AMF’s bad effects would probably have to flow through very minor population increases or fertility declines or something like that, and then a complex and probably weak causal chain from there to really big deal things. Whereas with AI safety work, which I think is typically very valuable in expectation, it also seems pretty easy to imagine it being quite bad.
E.g., extra support to it could create an attention hazard, highlighting the potential significance of AI, leading to more funding or government involvement, leading to faster development and less caution, etc. Or the safety research could “spill over” into capabilities development, which may not be negative at all, but plausibly could be substantially negative.
I don’t think this is an argument for avoiding longtermist interventions. This is because I think that, very roughly speaking, we should do what’s best in expectation, rather than worrying especially much about “keeping our hands clean” and avoiding any risk of causing harm. But this does seem a point worth noting, and I do think it’s an argument for thinking more about downside risks in relation to longtermist interventions than in relation to near-termist (e.g., global poverty) intervention.
(That’s perhaps more of a tangent than a response to your core claims.)
This just seems like a nonstarter. If our estimates of long-term effects are massively uncertain, how can they possibly be action-guiding?
(Minor point)
This is also my impression. And I think knowledge of that played some small role in my probably overdetermined shift towards longtermism.
But I’m also a bit concerned about the idea of using that trend as a factor in forming one’s own beliefs or decisions. I think we should be very cautious about doing so, and provide heavy caveats when discussing the idea of doing so. This is because I think it’s possible we could end up with an unhealthy combination of founders effects and information cascades, where some initial or vocal group happened to lean more a certain way; a bunch of people see that they did so, updates on that, and thus leans more that way; a bunch of people see that and do the same, etc.
(To be honest, I’m personally not so worried about this in relation to longtermism as a moral principle, but more in relation to claims about the future or which specific longtermist strategies to prioritise. I’m not sure exactly why this is. I think it’s largely that the latter seems to rely much more on a range of complex long-term forecasts that might be made fairly arbitrarily at some point but then become ossified into a community’s common sense.)
Yep I agree (I frame this as ‘beware updating on epistemic clones’ - people who have your beliefs for the same reason as you). My point in bringing this up was just that the common-sense view isn’t obviously near-termist.
(Minor, nit-picky point)
Here’s what you might have meant, which I’d endorse: “I’d love to see more instances of people trying to work out the flow through effects of global poverty interventions, and making decisions with those flow through effects as a very large factor.”
But the word “justifying” could imply the call here is just for people to keep on their current track, but switch the stated rationale. As you might be suggesting elsewhere, including by referencing Beware surprising and suspicious convergence, it seems plausible that some of this is happening, and that it’s not a good thing.
I think if my only choices were for people to keep on their current track for totally non-longtermist reasons or keep on their current track and say or convince themselves that it’s for longtermist reasons, I’d choose the former, because then they’ll probably do a better job of what they’re doing and would be less likely to end up with “bad epistemics”.
(See also The Bottom Line.)
Again, nice clarification.
I didn’t want to make any strong claims about which interventions people should end up prioritising, only about which effects they should consider to choose interventions.
Hi!
I think you mean to say: “every way a higher growth rate would be good is also an equally plausibly reason it would be bad”
Instead you wrote:
“Evidential symmetry here would be something like: every way a higher growth rate would be good is also an equally plausibly reason it would be good eg. increased emissions are equally likely to be good as they are to be bad.) ”
Thank you :) I’ve corrected it
I recommend the paper The Case for Strong Longtermism, as it covers and responds to many of these arguments in a precise philosophical framework.
“Our actions have dominating long-term effects that we cannot ignore.”
To me, this is a strange intuition. Most actions by most people most of the time disappear like ripples in a stream.
If this were not the case, reality would tear under the weight of schemes past people had for the present. Perhaps it is actually hard to change the course of history?
(Minor point)
Did you mean:
(1) “The urgency for direct work right now is greater in biorisk and short-timeline AI safety than in global poverty, animal welfare, or mental health, because of the greater the chance of lock-in in relation to biorisk and short-timeline AI safety”?
Or (2) “In biorisk and short-timeline AI safety, it’s more urgent to do direct work right now to avoid lock-in than to do long-term field-building and trajectory change”?
If you mean (1), I agree, and think that that’s a good point. (It doesn’t seem the case is 100% settled, but it seems to me clear enough to act on.)
If you mean (2), I think that’s less clear. I don’t disagree; I just don’t know. Since you’re specifying short-timeline AI safety, that does push in favour of direct work right now. But even a “short-timeline” might be decades, in which case field-building and trajectory change might be better. And biorisk may be with us for decades or centuries (perhaps partly dependent also on AI timelines).
(I hope to post soon about the matter of optimal timing for work or donations, outlining in a structured way (hopefully) all the key arguments and questions people have raised.)
Yep I meant (1) - thanks for checking. Also, that post sounds great—let me know if you want me to look over a draft :)
(Minor point) You write:
Regarding x-risk, did you mean:
(1) “Marginally increasing economic growth causes a (perhaps extremely slight) net increase in anthropogenic x-risk slightly, but this could be outweighed by other benefits unrelated to anthropogenic x-risk”?
Or (2): “Marginally increasing economic growth causes some things that increase in anthropogenic x-risk for some reasons, and some things that decrease it, and it’s unclear what the net effect on anthropogenic x-risk are?”
If you mean (2), I agree. If you meant (1), I think it’s quite unclear, but my current, quite uncertain guess is that marginally increasing economic growth probably slightly decreases x-risk on balance. (And this is almost entirely due to the effects on anthropogenic x-risk, since natural x-risks are probably far lower.) This guess is largely based on this paper’s conclusions (I lack the background to judge its arguments in detail).
Following that paper, I think growth might increase x-risk in the near-term (say ~100-200 years), and might decrease x-risk in the long-term (if the growth doesn’t come at the cost of later growth). I meant (1), but was thinking about the effect of x-risk in the near-term.
Great post, thank you.
If one accepts your conclusion, how does one go about implementing it? There is the work on existential risk reduction, which you mention. Beyond that, however, predicting any long-term effect seems to be a work of fiction. If you think you might have a vague idea of how things will turn out in 1k year, you must realize that even longer-term effects (1m? 1b?) dominate these. An omniscient being might be able to see the causal chain from our present actions to the far future, but we certainly cannot.
A question this raises for me is whether we should adjust our moral theories in any way. Given your conclusions, classic utilitarianism becomes a great idea that can never be implemented by us mere mortals. A bounded implementation, as MichaelStJules mentions, is probably preferable to ignoring utilitarianism completely, but that only answers this question by side-stepping it. I have come across philosophical work on “The Nonidentity Problem” which suggests that our moral obligations more or less extend to our grandchildren, but personaly I remain unconvinced by it.
I think there might be one area of human activity that, even given your conclusion, it is moral and rational to pursue—education. Not the contemporary kind which amounts to exercising our memories to pass standardized tests. More along the lines of what the ancient Greeks had in mind when they thought about education. The aim would be somewhere in the ballpark of producing critical thinking, compassionate, and physically fit people. These people will then be able to face the challenges they encounter, and which we cannot predict, in the best possible way. There is a real risk that humanity takes an unrecoverable turn for the worst, and while good education does not promise to prevent that, it increases the odds that we achieve the highest levels of human happiness and fulfillment as we set out to discover the farthest reaches of our galaxy.
I would love to hear your thoughts.
Pleased you liked it and thanks for the question. Here are my quick thoughts:
That kind of flourishing-education sounds a bit like Bostrom’s evaluation function described here: http://www.stafforini.com/blog/bostrom/
Or steering capacity described here: https://forum.effectivealtruism.org/posts/X2n6pt3uzZtxGT9Lm/doing-good-while-clueless
Unfortunately he doesn’t talk about how to construct the evaluation function, and steering capacity is only motivated by an analogy. I agree with you/Bostrom/Milan that there are probably some things that look more robustly good than others. It’s a bit unclear how to get these but something like :‘Build models of how the world works by looking to the past and then updating based on inside view arguments of the present/future. Then take actions that look good on most of your models’ seems vaguely right to me. Some things that look good to me are: investing, building the EA community, reducing the chance of catastrophic risks, spreading good values, getting better at forecasting, building models of how the world works
Adjusting our values based on them being difficult to achieve seems a bit backward to me, but I’m motivated by subjective preferences, and maybe it would make more sense if you were taking a more ethical/realist approach (eg. because you expect the correct moral theory to actually be feasible to implement).