Agree that X-risk is a better initial framing than longtermism—it matches what the community is actually doing a lot better. For this reason, I’m totally on board with “x-risk” replacing “longtermism” in outreach and intro materials. However, I don’t think the idea of longtermism is totally obsolete, for a few reasons:
Longtermism produces a strategic focus on “the last person” that this “near-term x-risk” view doesn’t. This isn’t super relevant for AI, but it makes more sense in the context of biosecurity. Pandemics with the potential to wipe out everyone are way worse than pandemics which merely kill 99% of people, and the ways we prepare for them seem likely to differ. On this view, bunkers and civilizational recovery plans don’t make much sense.
S-risks seem like they could very well be a big part of the overall strategy picture (even when not given normative priority and just considered as part of the total picture), and they aren’t captured by the short-term x-risk view.
The numbers you give for why x-risk might be the most important cause areas even if we ignore the long-term future, $30 million for a 0.0001% reduction in X-risk, don’t seem totally implausible. The world is big, and if you’re particularly pessimistic about changing it, then this might not be enough to budge you. Throw in an extra 10^30, though, and you’ve got a really strong argument, if you’re the kind of person that takes numbers seriously.
Submitting this now because it seems important, and I want to give this comment a chance to bubble to the top. Will fill in more reasons later if any major ones come up as I continue thinking.
S-risks seem like they could very well be a big part of the overall strategy picture (even when not given normative priority and just considered as part of the total picture), and they aren’t captured by the short-term x-risk view.
An existential risk is a risk that threatens the destruction of humanity’s long-term potential. But s-risks are worrisome not only because of the potential they threaten to destroy, but also because of what they threaten to replace this potential with (astronomical amounts of suffering).
I think the “short-term x-risk view” is meant to refer to everyone dying, and ignoring the lost long-term potential. Maybe s-risks could be similarly harmful in the short term, too.
Spreading wild animals to space isn’t bad for any currently existing humans or animals, so it isn’t counted under thoughtful short-termism or is discounted heavily. Same with a variety of S-risks (e.g. eventual stable totalitarian regime 100+ years out, slow space colonization, slow build up of Matrioshka brains with suffering simulations/sub-routines, etc.)
Oop, thanks for correction. To be honest I’m not sure what exactly I was thinking originally, but maybe this is true for non-AI S-risks that are slow, like spreading wild animals to space? I think this is mostly just false tho >:/
Agree that X-risk is a better initial framing than longtermism—it matches what the community is actually doing a lot better. For this reason, I’m totally on board with “x-risk” replacing “longtermism” in outreach and intro materials. However, I don’t think the idea of longtermism is totally obsolete, for a few reasons:
Longtermism produces a strategic focus on “the last person” that this “near-term x-risk” view doesn’t. This isn’t super relevant for AI, but it makes more sense in the context of biosecurity. Pandemics with the potential to wipe out everyone are way worse than pandemics which merely kill 99% of people, and the ways we prepare for them seem likely to differ. On this view, bunkers and civilizational recovery plans don’t make much sense.
S-risks seem like they could very well be a big part of the overall strategy picture (even when not given normative priority and just considered as part of the total picture), and they aren’t captured by the short-term x-risk view.
The numbers you give for why x-risk might be the most important cause areas even if we ignore the long-term future, $30 million for a 0.0001% reduction in X-risk, don’t seem totally implausible. The world is big, and if you’re particularly pessimistic about changing it, then this might not be enough to budge you. Throw in an extra 10^30, though, and you’ve got a really strong argument, if you’re the kind of person that takes numbers seriously.
Submitting this now because it seems important, and I want to give this comment a chance to bubble to the top. Will fill in more reasons later if any major ones come up as I continue thinking.
Why not?
An existential risk is a risk that threatens the destruction of humanity’s long-term potential. But s-risks are worrisome not only because of the potential they threaten to destroy, but also because of what they threaten to replace this potential with (astronomical amounts of suffering).
I think the “short-term x-risk view” is meant to refer to everyone dying, and ignoring the lost long-term potential. Maybe s-risks could be similarly harmful in the short term, too.
Spreading wild animals to space isn’t bad for any currently existing humans or animals, so it isn’t counted under thoughtful short-termism or is discounted heavily. Same with a variety of S-risks (e.g. eventual stable totalitarian regime 100+ years out, slow space colonization, slow build up of Matrioshka brains with suffering simulations/sub-routines, etc.)
Oop, thanks for correction. To be honest I’m not sure what exactly I was thinking originally, but maybe this is true for non-AI S-risks that are slow, like spreading wild animals to space? I think this is mostly just false tho >:/