Probably the most important “good things that can happen” after FAI are:
Whole brain emulation. It would allow eliminating death, pain and physical violence, not even mentioning ending discrimination and social stratification on the basis of appearance (although serious investment into cybersecurity would be required).
Full automation of the labor required to maintain a comfortable style of living for everyone. Avoiding a Malthusian catastrophe would still require reasonable reproduction culture (especially given immortality due to e.g. WBE).
It seems like the development of these would increase expected value massively in the medium term. I’m not sure what the effect on long term expected value would be (because we’d expect to develop these at some point anyway in the long term).
Good point. In the long run, the important thing is to reach the best attractor point / asymptotic trajectory. We need to develop much better understanding of the space of attractors (“possible ultimate fates”) and the factors leading to reaching one rather than another. I’d say that the things I mentioned are definitely on the wish list of the attractor we want to select.
Probably the most important “good things that can happen” after FAI are:
Whole brain emulation. It would allow eliminating death, pain and physical violence, not even mentioning ending discrimination and social stratification on the basis of appearance (although serious investment into cybersecurity would be required).
Full automation of the labor required to maintain a comfortable style of living for everyone. Avoiding a Malthusian catastrophe would still require reasonable reproduction culture (especially given immortality due to e.g. WBE).
It seems like the development of these would increase expected value massively in the medium term. I’m not sure what the effect on long term expected value would be (because we’d expect to develop these at some point anyway in the long term).
Good point. In the long run, the important thing is to reach the best attractor point / asymptotic trajectory. We need to develop much better understanding of the space of attractors (“possible ultimate fates”) and the factors leading to reaching one rather than another. I’d say that the things I mentioned are definitely on the wish list of the attractor we want to select.