Many EAs endorse two claims about existential risk. First, existential risk is currently high:
“(Existential Risk Pessimism) Per-century existential risk is very high.”
For example, Toby Ord (2020) puts the risk of existential catastrophe by 2100 at 1⁄6, and participants at the Oxford Global Catastrophic Risk Conference in 2008 estimated a median 19% chance of human extinction by 2100 (Sandberg and Bostrom 2008). Let’s ballpark Pessimism using a 20% estimate of per-century risk.
This isn’t really important for the implications of the post, but is it really true that many EAs endorse Existential Risk Pessimism (rather than something like Time of Perils)? I rarely see discussion of per-period existential risk estimates. The estimates you cite here are for existential risk this century (by 2100), not per century. I doubt those coming up with the figures you cite believe per century risk is about 20% on average (median), where this represents the probability of a new existential catastrophe (or multiple) starting in the given century, conditional on none before.
There was some discussion of per-period risks here and here.
Thanks Michael—you’ve found the terminological bane of my existence! (And the links are very helpful).
Of course it’s quite true that many Pessimists hold a Time of Perils View. One way to read the upshot of the paper is that Pessimists should probably hold a Time of Perils View.
The terminological problem is that, as you’ve said, my statement of Existential Risk Pessimism implies (or at least sneakily fails to rule out) that I’m talking about risk in all centuries, in which case Existential Risk Pessimism is strictly speaking incompatible with the Time of Perils View.
The natural solution to this problem is to let Existential Risk Pessimism be defined as a view about risk in this century. I tried that in the first draft. The problem I ran in to with this terminology is that I kept having to mention the assumption of constant risk throughout sections 2-3.3. And it made the paper very clunky to say this so many times. It also kinda ruined the punchline, because by the time I said the words “constant risk” a few times, everyone knew what the solution in Section 3.4 would be.
Readers complained so much about this that I changed it to the current terminology. As you mention, the current terminology is … a bit misleading.
I really don’t know how to fix this. I feel like both ways of defining Existential Risk Pessimism have drawn a lot of complaints from readers.
Surely there is a better third option, but i don’t know what it is. Do you have any thoughts?
I would just change the part of the intro I quoted to speak of this view as a possible view someone might hold, without attributing the view to Ord or those others currently cited, and without claiming many EAs actually hold it. If you could find others to attribute the view to, that would be good.
You could continue to cite the same people, but clearly distinguish the hypotheses: “While these are claims only about risks this century, what if a high risk probability persisted and compounded each century? Let’s call this hypothesis Existential Risk Pessimism:”
This isn’t really important for the implications of the post, but is it really true that many EAs endorse Existential Risk Pessimism (rather than something like Time of Perils)? I rarely see discussion of per-period existential risk estimates. The estimates you cite here are for existential risk this century (by 2100), not per century. I doubt those coming up with the figures you cite believe per century risk is about 20% on average (median), where this represents the probability of a new existential catastrophe (or multiple) starting in the given century, conditional on none before.
There was some discussion of per-period risks here and here.
Indeed! In The Precipice, Ord estimates a 50% chance that humanity never suffers an existential catastrophe (p.169).
Thanks Michael—you’ve found the terminological bane of my existence! (And the links are very helpful).
Of course it’s quite true that many Pessimists hold a Time of Perils View. One way to read the upshot of the paper is that Pessimists should probably hold a Time of Perils View.
The terminological problem is that, as you’ve said, my statement of Existential Risk Pessimism implies (or at least sneakily fails to rule out) that I’m talking about risk in all centuries, in which case Existential Risk Pessimism is strictly speaking incompatible with the Time of Perils View.
The natural solution to this problem is to let Existential Risk Pessimism be defined as a view about risk in this century. I tried that in the first draft. The problem I ran in to with this terminology is that I kept having to mention the assumption of constant risk throughout sections 2-3.3. And it made the paper very clunky to say this so many times. It also kinda ruined the punchline, because by the time I said the words “constant risk” a few times, everyone knew what the solution in Section 3.4 would be.
Readers complained so much about this that I changed it to the current terminology. As you mention, the current terminology is … a bit misleading.
I really don’t know how to fix this. I feel like both ways of defining Existential Risk Pessimism have drawn a lot of complaints from readers.
Surely there is a better third option, but i don’t know what it is. Do you have any thoughts?
I would just change the part of the intro I quoted to speak of this view as a possible view someone might hold, without attributing the view to Ord or those others currently cited, and without claiming many EAs actually hold it. If you could find others to attribute the view to, that would be good.
You could continue to cite the same people, but clearly distinguish the hypotheses: “While these are claims only about risks this century, what if a high risk probability persisted and compounded each century? Let’s call this hypothesis Existential Risk Pessimism:”