Feel free to message me on here.
JackM
Karma: 4,244
The analysis I linked to isn’t conclusive on longtermism being the clear winner if only considering the short-term. Under certain assumptions it won’t be the best. Therefore if only considering the short-term, many may choose not to give to longtermist interventions. Indeed this is what we see in the EA movement where global health still reigns supreme as the highest priority cause area.
What most longtermist analysis does is argue that if you consider the far future, longtermism then becomes the clear winner (e.g. here). In short, significantly more value is at stake with reducing existential risk because now you care about enabling far future beings to live and thrive. If longtermism is the clear winner then we shouldn’t see a movement that clearly prioritises global health, we should see a movement that clearly prioritises longtermist causes. This would be a big shift from the status quo.
As for your final point, I think I understand what you / the authors were saying now. I don’t think we have no idea what the far future effects of interventions like medical research are. We can make a general argument it will be good in expectation because it will help us deal with future disease which will help us reduce future suffering. Could that be wrong—sure—but we’re just talking about expectational value. With longtermist interventions, the argument is the far future effects are significantly positive and large in expectation. The simplest explanation is that future wellbeing matters, so reducing extinction risk seems good because we increase the probability of there being some welfare in the future rather than none.
existential risk can be justified without reference to the far future
This is pretty vague. If existential risk is roughly on par with other cause areas then we would be justified in giving any amount of resources to it. If existential risk is orders of magnitude more important then we should greatly prioritize it over other areas (at least on the current margin). So factoring in the far future does seem to be very consequential here.
FWIW I think it’s pretty unclear that something like reducing existential risk should be prioritised just based on near-term effects (e.g. see here). So I think factoring in that future people may value being alive and that they won’t want to be disempowered can shift the balance to reducing existential risk.
If future people don’t want to be alive they can in theory go extinct (this is the option value argument for reducing existential risk). The idea that future generations will want to be disempowered is pretty barmy, but again they can disempower themselves if they want to so it seems good to at least give them the option.
Hey, did you ever look into the Moral Consequences of Economic Growth book?
I’m not sure I buy the “We are not in a position to predict the best actions for the far future”
estimating the impact of current actions on medical research in 10,000 years—or even millions of years—is beyond our capacity.
I would say the following would, in expectation, boost medical research in millions of years:
Not going extinct or becoming disempowered: if you’re extinct or completely disempowered you can’t do medical research (and of course wellbeing would be zero or low!).
Investing in medical research now: if we invest in such research now we bring forward progress. So, in theory, in millions of years we would be ahead where we would have been if we had not invested now. If there’s a point at which we plateau with medical research then we we would just reach that plateau earlier and have more time to enjoy with the highest possible level of medical research.
We cannot reliably predict what future generations will value.
They will probably value:
Being alive: another argument for not going extinct.
Having the ability to do what they want: another argument for not becoming permanently disempowered. Or not to have a totalitarian regime control the world (e.g. through superintelligent AI).
Minimizing suffering: OK maybe they will like suffering who knows, but in my mind that would mean things have gone very wrong. Assuming they want to minimize suffering we should try to, for example, ensure factory farming does not spread out to other planets and therefore persist for millenia. Or advocate for the moral status of digital minds.
It’s slightly odd this paper argues that:
The urgency of addressing existential risks does not depend on the far future; the importance of avoiding these risks is clear when focusing on the present and the next few generations.
But then also says:
Focusing on the far future comes at a cost to addressing present-day needs and crises, such as health issues and poverty.
I’m left uncertain if the authors are in favor of spending to address existential risk, which would of course lead to less money to address present-day suffering due to health issues and poverty.
Yes but if I were to ask my non-EA friends what they give to (if they give to anything at all) they will say things like local educational charities, soup kitchens, animal shelters etc. I do think EA generally has more of a focus on saving lives.
You’re right that the issue at its core isn’t the meat eater problem. The bigger issue is that we don’t even know if saving lives now will increase or decrease future populations (there are difficult arguments on both sides). If we don’t even know that, then we are going to be at a complete loss to try to conduct assessments on animal welfare and climate change, even though we know there are going to be important impacts here.
I dispute this. I’m admittedly not entirely sure but here is my best explanation.
A lot of EA interventions involve saving lives which influences the number of people who will live in the future. This in turn, we know, will influence the following (to just give two examples):
The number of animals who will be killed for food (i.e. impacting animal welfare).
CO2 emissions and climate change (i.e. impacting the wellbeing of humans and wild animals in the future).
Importantly, we don’t know the sign and magnitude of these “unintended” effects (partly because we don’t in fact know if saving lives now causes more or fewer people in the future). But we do know that these unintended effects will predictably happen and that they will swamp the size of the “intended” effect of saving lives. This is where the complex cluelessness comes in. Considering predictable effects (both intended and not intended), we can’t really weigh them. If you think you can weigh them, then please tell me more.
So I think it’s the saving lives that really gets us into a pickle here—it leads to so much complexity in terms of predictable effects.
There are some EA interventions that don’t involve saving lives and don’t seem to me to run into a cluelessness issue e.g. expanding our moral circle through advocacy, building AI governance structures to (for instance) promote global cooperation, global priorities research. I don’t think these interventions run into the complex cluelessness issue because, in my opinion, it seems easy to say that the expected positives outweigh expected negatives. I explain this a little more in this comment chain.
Also, note that under Greaves’ model there are types of cluelessness that are not problematic, which she calls “simple cluelessness”. An example is if we are deciding whether to conceive a child on a Tuesday or a Wednesday. Any chance that one of the options might have some long-run positive or negative consequence will be counterbalanced by an equal chance that the other will have that consequence. In other words there is evidential symmetry across the available choices.
A lot of “non-EA” altruistic actions I think we will have simple cluelessness about (rather than complex), in large part because they don’t involve saving lives and are often on quite a small scale so aren’t going to predictably influence things like economic growth. For example, giving food to a soup kitchen—other than helping people who need food it isn’t at all predictable what other unintended effects will be so we have evidential symmetry and can ignore them. Basically, a lot of “non-EA” altruistic actions might not have predictable unintended effects, in large part because they don’t involve saving lives. So I don’t think they will run us into the cluelessness issue.
I need to think about this more but would welcome thoughts.
I see where Greaves is coming from with the longtermist argument. One way to avoid the complex cluelessness she describes is to ensure the direct/intended expected impact of your intervention is sufficiently large so as to swamp the (forseeable) indirect expected impacts. Longtermist interventions target astronomical / very large value, so they can in theory meet this standard.
I’m not claiming all longtermist interventions avoid the cluelessness critique. I do think you need to consider interventions on a case-by-case basis. But I think there are some fairly general things we can say. For example, the issue with global health interventions is that they pretty much all involve increasing the consumption of animals by saving human lives, so you have a negative impact there which is hard to weigh against the benefits of saving a life. You don’t have this same issue with animal welfare interventions.
Just to make sure I understand your position—are you saying that the cluelessness critique is valid and that it affects all altruistic actions? So Effective Altruism and altruism generally are doomed enterprises?
I don’t buy that we are clueless about all actions. For example, I would say that something like expanding our moral circle to all sentient beings is robustly good in expectation. You can of course come up with stories about why it might be bad, but these stories won’t be as forceful as the overall argument that a world that considers the welfare of all beings (that have the capacity for welfare) important is likely better than one that doesn’t.
I’d also add the cluelessness critique as relevant reading. I think it’s a problem for global health interventions, although realize that one could also argue that it is a problem for animal welfare interventions. In any case it seems highly relevant for this debate.
Thanks for taking a balanced view, but I would have liked to see more discussion of the replaceability argument which really is pivotal here.
You say that whoever is hired into a progress-accelerating role, even if they are safety-conscious, will likely be most effective in the role and so will accelerate progress more than an alternative candidate. This is fair but may not be the whole story. Could the fact that they are safety-conscious mean they can develop the AI in a safer way than the alternative candidate? Maybe they would be inclined to communicate and cooperate more with the safety teams than an alternative candidate. Maybe they would be more likely to raise concerns to leadership etc.
If these latter effects dominate it could be worth suggesting that people in the EA community apply even for progress-accelerating roles, and it could be more important for them to take roles at less reliable places like OpenAI than slightly more reliable like Anthropic.
The link with the comprehensive list of organizations and projects that engage in depolarization doesn’t work. Here is the link: Organizations Bridging Divides | Resources | The Morton Deutsch International Center for Cooperation and Conflict Resolution | Teachers College, Columbia University
And the pdf: Organizations-Bridging-Divides-Nov-2020.pdf (columbia.edu)
The person who gets the role is obviously going to be highly intelligent, probably socially adept, and highly-qualified with experience working in AI etc. etc. OpenAI wouldn’t hire someone who wasn’t.
The question is do you want this person also to care about safety. If so I would think advertising on the EA job board would increase the chance of this.
If you think EAs or people who look at the 80K Hours job board are for some reason less good epistemically than others then you will have to explain why because I believe the opposite.
You’re referring to job boards generally but we’re talking about the 80K job board which is no typical job board.
I would expect someone who will do a good job to be someone going in wanting to stop OpenAI destroying the world. That seems to be someone who would read the 80K Hours job board. 80K is all about preserving the future.
They of course also have to be good at navigating organizational social scenes while holding onto their own epistemics which in my opinion are skills commonly found in the EA community!
I think the evidence we have from OpenAI is that it isn’t very helpful to “be a safety conscious person there
It’s insanely hard to have an outsized impact in this world. Of course it’s hard to change things from inside OpenAI, but that doesn’t mean we shouldn’t try. If we succeed it could mean everything. You’re probably going to have lower expected value pretty much anywhere else IMO, even if it does seem intractable to change things at OpenAI.
I think it’s especially not helpful if you’re a low-context person, who reads an OpenAI job board posting, and isn’t going in with a specific plan to operate in an adversarial environment.
Surely this isn’t the typical EA though?
If OpenAI doesn’t hire an EA they will just hire someone else. I’m not sure if you tackle this point directly (sorry if I missed it) but doesn’t it straightforwardly seem better to have someone safety-conscious in these roles rather than someone who isn’t safety-conscious?
To reiterate, it’s not like if we remove these roles from the job board that they will less likely be filled. They would still definitely be filled, just by someone less safety-conscious in expectation. And I’m not sure the person who would get the role would be “less talented” in expectation because there are just so many talented ML researchers—so I’m not sure removing roles from the job board would slow down capabilities development much if at all.
I get a sense that your argument is somewhat grounded in deontology/virtue ethics (i.e. “don’t support a bad organization”) but perhaps not so much in terms of consequentialism?
Yeah the strong longtermism paper elucidates this argument. I also provide a short sketch of the argument here. At its core is the expected vastness of the future that allows longtermism to beat other areas. The argument for “normal” longtermism i.e. not “strong” is pretty much the same structure.
Yes that’s true. Again we’re dealing with expectations and most people expect the future to be good if we manage not to go extinct. But it’s also worth noting that reducing extinction risk is just one class of reducing existential risk. If you think the future will be bad, you can work to improve the future conditional on us being alive or, in theory, you can work to make us go extinct (but this is of course a bit out there). Improving the future conditional on us being alive might involve tackling climate change, improving institutions, or aligning AI.
And, to reiterate, while we focus on these areas to some extent now, I don’t think we focus on them as much as we would in a world where society at large accepts longtermism.