Extinction being bad assumes that our existence in the future is a net positive. There’s the possibility for existence to be net negative, in which case extinction is more like a zero point.
On the one hand, negativity bias means that all other things being equal, suffering tends to outweigh equal happiness. On the other hand, there’s a kind of progress bias where sentient actors in the world tend to seek happiness and avoid suffering and gradually make the world better.
Thus, if you’re at all optimistic that progress is possible, you’d probably assume that the future will be net positive in the very long run.
So, I sentimentally lean towards thinking that a net negative future is less likely than a net positive one, but, given tremendous uncertainty about the future, I would consider it more rational to apply something like the Principle of Maximum Entropy, and set our priors to each possibility being equally likely.
If net positive, extinction, and net negative scenarios are equally likely, than the negative value of the net negative scenarios should outweigh the relatively neutral value of extinction scenarios, and so we should put more emphasis on preventing these scenarios.
Though, I don’t really like this being a forced dichotomy. Working to prevent both to some degree as a form of cause area portfolio diversification is probably a better way to manage the risk.
Extinction being bad assumes that our existence in the future is a net positive. There’s the possibility for existence to be net negative, in which case extinction is more like a zero point.
On the one hand, negativity bias means that all other things being equal, suffering tends to outweigh equal happiness. On the other hand, there’s a kind of progress bias where sentient actors in the world tend to seek happiness and avoid suffering and gradually make the world better.
Thus, if you’re at all optimistic that progress is possible, you’d probably assume that the future will be net positive in the very long run.
So, I sentimentally lean towards thinking that a net negative future is less likely than a net positive one, but, given tremendous uncertainty about the future, I would consider it more rational to apply something like the Principle of Maximum Entropy, and set our priors to each possibility being equally likely.
If net positive, extinction, and net negative scenarios are equally likely, than the negative value of the net negative scenarios should outweigh the relatively neutral value of extinction scenarios, and so we should put more emphasis on preventing these scenarios.
Though, I don’t really like this being a forced dichotomy. Working to prevent both to some degree as a form of cause area portfolio diversification is probably a better way to manage the risk.