“Is this risk actually existential?” may be less important than we think
[Epistemic status: fast written post of something I have basically always felt but never really put into words. It still is a half-baked idea and it is really simple, but I don’t think I ever came across reading anything similar]
EA and adjacent movements puts a lot of attention into distinguish whether a risk is actually an x-risk or not. Of course, the difference in the outcomes between a fulfilled x-risk and a fulfilled not-x-risk are not of grade but of class: humanity (and its potential) exists or not.
I think the fact that the difference between x-risks and the other risks is of class, makes EAs spend way too much energy and time assessing whether a risk is actually existential or not. I don’t think this is time well spend. In general, if somebody has to put a good deal of effort and research to assess this, it means that the risk is a hell of a risk, and the incentives to minimise it, most likely maximal. [I am referring to, basically, close calls here, but I can imagine myself expanding it to global catastrophic risks, for example.]
In practical terms, these differences seem to me almost irrelevant. Does it make any difference in the actions anyone would possibly want to take to mitigate an extreme risk whether this risk is actually existential or not? For example: Does it make any difference whether a non-alligned superintelligent AGI will actively try to kill all humanity or not? If we are certain that it won’t, we would still live in a world where we are the ants and it is humanity. Even if we think we eventually could climb out of our ‘ant state’ to a state with more potential for humanity, should we really put less effort in mitigating this risk than if we’d think the AGI will eliminate us? It would feel very odd to me to answer yes to this question. [edited to add the following:] The reality is that resources (all, from money, to energy, effort or time) are finite and not enough are devoted to mitigate very large risks in general. Until this changes, whether a risk is actually existential or not seems to me much less important than EA as movement think.
In another level, there is also the issue that such nitpicks generate a lot of debate and are often difficult to fully understand by the general public. More often that we would like, these debates contribute to the growing wave against EA, since from the outside this can look like some nerds having fun / wasting time and calling themselves “effective” for doing so.
[I wrote this post pretty fast and almost in one go. Please, tell me if there is something that is not understandable, improvable, or wrong and I will try to update accordingly]
This misunderstands what an existential risk is, at least as used by the philosophers who’ve written about this. Nick Bostrom, for example, notes that the extinction of humanity is not the only thing that counts as an extinction risk. (The term “existential risk” is unfortunately a misnomer in this regard.) Something that drastically curtails the future potential of humanity would also count.
;-)
I’m not sure I understand your point then...
Surely a future in which humanity flourishes into the longterm future is a better one than a future where people are living as “ants.” And if we have uncertainty about which path we’re on and there are plausible reasons to think we’re on the ant path, it can be worthwhile to figure that out so we can shift in a better direction.
Exactly. Even if the ant path may not be permanent, ie. if we could climb out of it.
My point is that, in terms of the effort I would like humanity to devote to minimise this risk, I don’t think it makes any difference whether the ant state is strictly permanent or we could eventually get out of it. Maybe if it were guaranteed to get out of it or even “only” very likely that we could get out of this ant state I could understand devoting less effort in mitigating this risk than if we’d think the AGI will eliminate us (or the ant state would be unescapable).
If we agree on this, the fact that a risk is actually existential or not is in practice close to irrelevant.
Maybe a more realistic example would be helpful here. There have been recent reports claiming that, although it will negatively affect millions of people, climate change is unlikely to be an existential risk. Suppose that’s true. Do you think EAs should devote as much time and effort preventing climate change-level risks as they do preventing existential risks?
Let’s speak about humanity in general and not about EAs, cause where EA focus does not only depend on the degree of the risk.
Yes, I don’t think humanity should currently devote less efforts to prevent such risks than x-risks. Probably the point is that we are doing way too less to tackle dangerous non-immediate risks in general, so it does not make any practical difference whether the risk is existential or only almost existential. And this point of view does not seem controversial at all, it is just not explicitly stated. It is not just not-EAs that are devoting a lot of effort to prevent climate change, an increasing fraction of EAs do as well.
I suppose I agree that humanity should generally focus more on catastrophic (non-existential) risks.
That said, I think this is often stated explicitly. For example, MacAskill in his recently book explicitly says that many of the actions we take to reduce x-risks will also look good even for people with shorter-term priorities.
Do you have any quote from someone who says we shouldn’t care about catastrophic risks at all?
I’m not saying this. And I really don’t see how you came to think I do.
The only thing I say is that I don’t see how anyone would argue that humanity should devote less effort to mitigate a given risk just because it turns out that it is not actually existential even though it may be more than catastrophic. Therefore, finding out if a risk is actually existential or not is not really valuable.
I’m not saying anything new here, I made this point several times above. Maybe it is not very clearly done, but I don’t really know how to state it differently.