Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
https://reducing-suffering.org/near-miss/
Just gonna boost this excellent piece by Tomasik. I think partial alignment/near-misses causing s-risk is potentially an enormous concern. This is more true the shorter timelines are and thus the more likely people are to try using “hail mary” risky alignment techniques. Also more true for less principled/Agent Foundations-type alignment directions.
Can someone provide a more realistic example of partial alignment causing s-risk than SignFlip or MisconfiguredMinds? I don’t see either of these as something that you’d be reasonably likely to get by say, only doing 95% of the alignment research necessary rather than 110%.
Brian Tomasik wrote something similar about the risks of slightly misaligned artificial intelligence, although it is focused on suffering risks specifically rather than on existential risks in general.
I want a word which covers {x-risk, s-risk}, “Existential or worse”.
Doom?
“x-risk” covers “x-risk or worse” right?
Yes, I’d say so.
I guess that might raise the question if there is a term specifically for x-risks that aren’t s-risks. My sense is that people often use the term “x-risk” for that concept as well, but in some contexts one might want to have another term; to distinguish the two concepts.
I always thought s-risks are a subset of x-risks, e.g. that’s how CLR framed it here:
https://longtermrisk.org/s-risks-talk-eag-boston-2017/
Basic argument seems to be: Permanent astronomical hell is also curtailment of humanity’s potential, one that is very high in the dimensions of scope (astronomical) and intensity (involves hellish levels of suffering).
Good framing, but I’m surprised they went for it since it partially obscures S behind its larger more popular brother X.
One explanation might be that historically there seemed to have been somewhat of a divide between people worrying about s-risks and x-risks (which were ~ suffering-focused and ~ classic utilitarians), and this framing might’ve helped getting more cooperation started.
“At least existential”
Gotta be one word or bust