I have thought about similar issues as in your article and I my conclusions are broadly the same. Unfortunately, I have not written anything down since thinking about longtermism is something I do beside my job and family. I have some quick remarks:
Your conclusions in Section 6 are in my opinion pretty robust, even if you use a more general mathematical framework.
It is very unclear if space colonization increases or decreases existential risk. The main reason is that it is probably technologically feasible to send advanced weapons across astronomical distances, while building trust across such distances is hard.
Solving the AI alignment problem helps, but you need an exceptionally well aligned AI to realize the “time of perils”-scenario. If an AI does not “kill everyone immediately”, it is not clear if it is able to stick to positive human values for several million years and can coordinate with AIs in space colonies which may have different values.
Since I have seen so many positive reactions to your article, I am wondering if it has some impact if I try to find time to write more about my thoughts.
I have thought about similar issues as in your article and I my conclusions are broadly the same. Unfortunately, I have not written anything down since thinking about longtermism is something I do beside my job and family. I have some quick remarks:
Your conclusions in Section 6 are in my opinion pretty robust, even if you use a more general mathematical framework.
It is very unclear if space colonization increases or decreases existential risk. The main reason is that it is probably technologically feasible to send advanced weapons across astronomical distances, while building trust across such distances is hard.
Solving the AI alignment problem helps, but you need an exceptionally well aligned AI to realize the “time of perils”-scenario. If an AI does not “kill everyone immediately”, it is not clear if it is able to stick to positive human values for several million years and can coordinate with AIs in space colonies which may have different values.
Since I have seen so many positive reactions to your article, I am wondering if it has some impact if I try to find time to write more about my thoughts.