First, thank you for the informative post.
they claim that we’re closer to nuclear war than any time since the Cuban Missile Crisis, which is clearly nonsense
John Mearsheimer also believes P(nuclear war) is higher now that at any time during the Cold War. If you like, I can try to find where he says that in a video of one of his speeches and interviews.
His reasoning is that the US national-security establishment has become much less competent since the decisive events of the Cold War with the result that Russia is much more likely to choose to start a nuclear war than they would be if the security situation the US currently finds itself in were being managed by the US’s Cold-War leaders. I’m happy to elaborate if there is interest.
There are 2 concurrent research programs, and if one program (capability) completes before the other one (alignment), we all die, but the capability program is an easier technical problem than the alignment program. Do you disagree with that framing? If not, then how does “research might proceed faster than we expect” give you hope rather than dread?
Also, I’m guessing you would oppose a worldwide ban starting today on all “experimental” AI research (i.e., all use of computing resources to run AIs) till the scholars of the world settle on how to keep an AI aligned through the transition to superintelligence. That’s my guess, but please confirm. In your answer, please imagine that the ban is feasible and in fact can be effective (“leak-proof”?) enough to give the AI theorists all they time they need to settle on a plan even if that takes many decades. In other words, please indulge me this hypothetical question because I suspect it is a crux.
“Settled” here means that a majority of non-senile scholars / researchers who’ve worked full-time on the program for at least 15 years of their lives agree that it is safe for experiments to start as long as they adhere to a particular plan (which this majority agree on). Kinda like the way scholars have settled on the conclusion that anthropogenic emissions of carbon dioxide are causing the Earth to warm up.
I realize that there are safe experiments that would help the scholars of the world a lot in their theorizing about alignment, but I also expect that the scholars / theorist could probably settle on a safe effective plan without the benefit of experiments beyond the experiments that have been run up to now even though it might take them longer than it would with the benefit. Do you agree?