TL;DR are there any forum posts or similarly accessible writing that clarify different notions of x-risk? If not, does it seem worth writing?
My impression is that prevailing notions of x-risk (i.e. what it means, not specific cause areas) have broadened or shifted over time, but there’s a lack of clarity about what notion/definition people are basing arguments on in discourse.
At the same time, discussion of x-risk sometimes seems too narrow. For example, in the most recent 80K podcast with Will MacAskill, they at one point talk about x-risk in terms of literal 100% human annihilation. IMO this is one of the least relevant notions of x-risk, for cause prioritisation purposes. Perhaps there’s a bias because literal human extinction is the most concrete/easy to explain/easy to reason about? Nowadays I frame longtermist cause prioritisation more like “what could cause the largestlosses to the expected value of the future” than “what could plausibly annihilate humanity”.
Bostrom (2002) defined x-risk as “one where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential”. There is also a taxonomy in section 3 of the paper. Torres (2019) explains and analyses five different definitions of x-risk, which I think all have some merit.
To be clear I think many people have internalised broader notions of x-risk in their thoughts and arguments, both generally and for specific cause areas. I just think it could use some clarification and a call for people to clarify themselves, e.g. in a forum post.
I’d love to see this post and generally more discussion of what kinds of x-risks and s-risks matter most. 80K’s views seem predicated on deeply held, nuanced, and perhaps unconventional views of longtermism, and it can be hard to learn all the context to catch up on those discussions.
I definitely think it could be good to have a set of definitions written up on the Forum, especially if it’s briefer/easier to reference than the Torres paper or other academic papers defining X-risk. If you do end up writing something, I’d be happy to look it over before you publish!
TL;DR are there any forum posts or similarly accessible writing that clarify different notions of x-risk? If not, does it seem worth writing?
My impression is that prevailing notions of x-risk (i.e. what it means, not specific cause areas) have broadened or shifted over time, but there’s a lack of clarity about what notion/definition people are basing arguments on in discourse.
At the same time, discussion of x-risk sometimes seems too narrow. For example, in the most recent 80K podcast with Will MacAskill, they at one point talk about x-risk in terms of literal 100% human annihilation. IMO this is one of the least relevant notions of x-risk, for cause prioritisation purposes. Perhaps there’s a bias because literal human extinction is the most concrete/easy to explain/easy to reason about? Nowadays I frame longtermist cause prioritisation more like “what could cause the largest losses to the expected value of the future” than “what could plausibly annihilate humanity”.
Bostrom (2002) defined x-risk as “one where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential”. There is also a taxonomy in section 3 of the paper. Torres (2019) explains and analyses five different definitions of x-risk, which I think all have some merit.
To be clear I think many people have internalised broader notions of x-risk in their thoughts and arguments, both generally and for specific cause areas. I just think it could use some clarification and a call for people to clarify themselves, e.g. in a forum post.
I’d love to see this post and generally more discussion of what kinds of x-risks and s-risks matter most. 80K’s views seem predicated on deeply held, nuanced, and perhaps unconventional views of longtermism, and it can be hard to learn all the context to catch up on those discussions.
One distinction I like is OpenPhil talking about Level 1 and Level 2 GCRs: https://www.openphilanthropy.org/blog/long-term-significance-reducing-global-catastrophic-risks
I definitely think it could be good to have a set of definitions written up on the Forum, especially if it’s briefer/easier to reference than the Torres paper or other academic papers defining X-risk. If you do end up writing something, I’d be happy to look it over before you publish!
Some discussion here, too, in the context of introducing s-risks:
https://foundational-research.org/s-risks-talk-eag-boston-2017/