Now, the second issue: “As for my proposed “alternative exit strategy”, I agree with you that civilization as it stands today seems woefully inadequate to safely handle either nuclear weapons or advanced AI technology for very long. Personally I am optimistic about trying to create new, experimental institutions (like better forms of voting, or governments run in part by prediction markets) that could level-up civilization’s adequacy/competence and create a wiser civilization better equipped to handle these dangerous technologies.”
First of all, I consider myself to have some expert knowledge about the “human alignment” problem, and precisely this post is mainly about my reasons for pessimism (either rational and historic-intuitive). Even if macro trends were uniformly positive (which they are not), human governance systems are simply not reliable enough. And in any case if you are lucky enough to create an ideal republic, let’s say in Switerzerland (they are almost there!), how would you “export” it to Russia, China and North Korea? It is not only the problem of designing governance systems. You need to deploy your system against entrenched and ruthless political elites.
I agree that hoping for ideal societies is a bit of a pipe dream. But there is some reason for hope. China and Russia, for instance, were both essentially forced to abandon centrally-planned economies and adopt some form of capitalism in order to stay competitive with a faster-growing western world. Unfortunately, the advantages of democracy vs authoritarianism (although there are many) don’t seem quite as overwhelming as the advantages of capitalism vs central planning. (Also, if you are the authoritarian in charge, maybe you don’t mind switching economic systems, but you probably really want to avoid switching political systems!)
Alternatively, if Russia/China/etc didn’t reform, I would expect them to eventually fall further and further behind (like North Korea vs South Korea); they’d still have nukes of course, but after many decades of falling behind, eventually I’d expect the US could field some technology—maybe aligned AI like you say, or maybe just really effective missile-defence—that would help end the era of nuclear risk (or at least nuclear risk from backwards, fallen-behind countries).
Now, the second issue: “As for my proposed “alternative exit strategy”, I agree with you that civilization as it stands today seems woefully inadequate to safely handle either nuclear weapons or advanced AI technology for very long. Personally I am optimistic about trying to create new, experimental institutions (like better forms of voting, or governments run in part by prediction markets) that could level-up civilization’s adequacy/competence and create a wiser civilization better equipped to handle these dangerous technologies.”
First of all, I consider myself to have some expert knowledge about the “human alignment” problem, and precisely this post is mainly about my reasons for pessimism (either rational and historic-intuitive). Even if macro trends were uniformly positive (which they are not), human governance systems are simply not reliable enough. And in any case if you are lucky enough to create an ideal republic, let’s say in Switerzerland (they are almost there!), how would you “export” it to Russia, China and North Korea? It is not only the problem of designing governance systems. You need to deploy your system against entrenched and ruthless political elites.
I agree that hoping for ideal societies is a bit of a pipe dream. But there is some reason for hope. China and Russia, for instance, were both essentially forced to abandon centrally-planned economies and adopt some form of capitalism in order to stay competitive with a faster-growing western world. Unfortunately, the advantages of democracy vs authoritarianism (although there are many) don’t seem quite as overwhelming as the advantages of capitalism vs central planning. (Also, if you are the authoritarian in charge, maybe you don’t mind switching economic systems, but you probably really want to avoid switching political systems!)
But maybe if the West developed even better governing institutions (like “futarchy”, a form of governance based partly on democratic voting and partly on prediction markets), or otherwise did a great job of solving our own problems (like doing a good job generating lots of cheap, clean energy, or adopting Georgist and Yimby policies to lower the cost of housing and threreby boost the growth of western economies), once again we might pressure our geopolitical competitors to reform their own institutions in order to keep up.
Alternatively, if Russia/China/etc didn’t reform, I would expect them to eventually fall further and further behind (like North Korea vs South Korea); they’d still have nukes of course, but after many decades of falling behind, eventually I’d expect the US could field some technology—maybe aligned AI like you say, or maybe just really effective missile-defence—that would help end the era of nuclear risk (or at least nuclear risk from backwards, fallen-behind countries).
I agree with all your political positions! Let’s run for office together!
Now, more seriously, thank you very much for the links.