I think I would have found Ajeya’s cold takes guest post on “Why AI alignment could be hard with modern deep learning” persuasive back when I was skeptical. It is pretty short. I think the reason why I didn’t find what you call “short stories by Eliezer Yudkowsky” persuasive was because they tended to not use concepts / terms from ML. I guess even stuff like orthogonality thesis and instrumental convergence thesis was not that convincing to me on a gut level even though I didn’t disagree with the actual argument for them because I had the intuition that whether misaligned AI was a big deal depended on details of how ML actually worked, which I didn’t know. To me back then it looked like most people I knew with much more knowledge of ML were not concerned about AI x-risk so probably it wasn’t a big deal.
I think I would have found Ajeya’s cold takes guest post on “Why AI alignment could be hard with modern deep learning” persuasive back when I was skeptical. It is pretty short. I think the reason why I didn’t find what you call “short stories by Eliezer Yudkowsky” persuasive was because they tended to not use concepts / terms from ML. I guess even stuff like orthogonality thesis and instrumental convergence thesis was not that convincing to me on a gut level even though I didn’t disagree with the actual argument for them because I had the intuition that whether misaligned AI was a big deal depended on details of how ML actually worked, which I didn’t know. To me back then it looked like most people I knew with much more knowledge of ML were not concerned about AI x-risk so probably it wasn’t a big deal.