Just spitballing, but I spontaneously don’t find it completely unrealistic. Would a decades long moratorium on transformative AI count as a long reflection, which is prolonged until some consensus forms among world governments? That currently seems unthinkable, but if the right people in the US and Chinese governments would be convinced, and we would have lived through a global catastrophe due to misaligned AI systems, I wouldn’t be surprised to see a global coordination to control advances in AI and scaling up alignment and ethical reflection research, research which would have significant sway over the future of humanity over decades.
I’m spontaneosly also not convinced that this moratorium would pose so significant costs on many relevant actors. Progress would continue, just slower. I.e. we would still eradicate poverty in the coming decades, make significant progress on fighting most major diseases, maybe even have just-not-quite transformative personal AI assistants, etc. Right?
Just spitballing, but I spontaneously don’t find it completely unrealistic. Would a decades long moratorium on transformative AI count as a long reflection, which is prolonged until some consensus forms among world governments? That currently seems unthinkable, but if the right people in the US and Chinese governments would be convinced, and we would have lived through a global catastrophe due to misaligned AI systems, I wouldn’t be surprised to see a global coordination to control advances in AI and scaling up alignment and ethical reflection research, research which would have significant sway over the future of humanity over decades.
I’m spontaneosly also not convinced that this moratorium would pose so significant costs on many relevant actors. Progress would continue, just slower. I.e. we would still eradicate poverty in the coming decades, make significant progress on fighting most major diseases, maybe even have just-not-quite transformative personal AI assistants, etc. Right?