The real problem is that in large scale problems like AI safety, progress is usually continuous, not discrete. This we can talk about partial alignment problems, which realistically is the best EA/LessWrong can do. I don’t expect them to ever be able to get AI to be particularly moral or not destabilize society, but existential catastrophe is likely to be avoided.
Also, I’m going to steal part of Vaidehi Agarwalla’s comment and improve upon it here:
Your post links to 2 articles from Eliezer Yudkowsky’s / MIRI’s perspective of AI alignment, which is a (but importantly, not the only) perspective of alignment research that is an outlier in it’s direness. We have good reason to believe that this caused by unnecessary discreteness in their framing of the AI Alignment problem.
The real problem is that in large scale problems like AI safety, progress is usually continuous, not discrete. This we can talk about partial alignment problems, which realistically is the best EA/LessWrong can do. I don’t expect them to ever be able to get AI to be particularly moral or not destabilize society, but existential catastrophe is likely to be avoided.
Also, I’m going to steal part of Vaidehi Agarwalla’s comment and improve upon it here:
Your post links to 2 articles from Eliezer Yudkowsky’s / MIRI’s perspective of AI alignment, which is a (but importantly, not the only) perspective of alignment research that is an outlier in it’s direness. We have good reason to believe that this caused by unnecessary discreteness in their framing of the AI Alignment problem.