The stakes surrounding ASI are extremely high, to the point that we canāt be confident that humanity would long survive this development
I guess the risk of human extinction over the next 10 years is 10^-7, and I am not aware of any quantitative empirical modelling suggesting otherwise. I do not think that is high enough to justify your conclusion that āAI risk warrants urgent further investigation and precautionary measuresā, even accounting for longterm effects. I very much agree there should be some investigation and precautionary measures, but I do not consider this āurgentā at the margin.
Thanks for crossposting this, Richard!
I guess the risk of human extinction over the next 10 years is 10^-7, and I am not aware of any quantitative empirical modelling suggesting otherwise. I do not think that is high enough to justify your conclusion that āAI risk warrants urgent further investigation and precautionary measuresā, even accounting for longterm effects. I very much agree there should be some investigation and precautionary measures, but I do not consider this āurgentā at the margin.