create a treaty for countries to sign that ban research into AGI.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
Still, I think you are really overestimating the ability of the president to move the scientific community. For instance, we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all. Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism. But the point still stands.
They may not have budged climate scientists, but there other ways they may have influenced policy. Did they (or other partisans) alter the outcomes of Washington Initiative 1631 or 732? That seems hard to evaluate.
Yes, policy can be changed for sure. I was just referring to actually changing minds in the community, as he said—“Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit.”
we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all.
I have updated in your direction.
Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism.
Yep.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
No I meant starting today. My impression is that coalition-building in Washington is tedious work. Scientists agreed to avoid gene editing in humans well before it was possible (I think). In part, that might have made it easier since the distantness of it meant fewer people were researching it to begin with. If AGI is a larger part of an established field, it seems much harder to build a consensus to stop doing it.
FWIW I don’t think that would be a good move. I don’t feel like fully arguing it now, but main points (1) sooner AGI development could well be better despite risk, (2) such restrictions are hard to reverse for a long time after the fact, as the story of human gene editing shows, (3) AGI research is hard to define—arguably, some people are doing it already.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
Still, I think you are really overestimating the ability of the president to move the scientific community. For instance, we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all. Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism. But the point still stands.
They may not have budged climate scientists, but there other ways they may have influenced policy. Did they (or other partisans) alter the outcomes of Washington Initiative 1631 or 732? That seems hard to evaluate.
Yes, policy can be changed for sure. I was just referring to actually changing minds in the community, as he said—“Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit.”
I have updated in your direction.
Yep.
No I meant starting today. My impression is that coalition-building in Washington is tedious work. Scientists agreed to avoid gene editing in humans well before it was possible (I think). In part, that might have made it easier since the distantness of it meant fewer people were researching it to begin with. If AGI is a larger part of an established field, it seems much harder to build a consensus to stop doing it.
FWIW I don’t think that would be a good move. I don’t feel like fully arguing it now, but main points (1) sooner AGI development could well be better despite risk, (2) such restrictions are hard to reverse for a long time after the fact, as the story of human gene editing shows, (3) AGI research is hard to define—arguably, some people are doing it already.