The other thing is that in 20 years, we might want the president on the phone with very specific proposals. What are the odds they’ll spend a weekend discussing AGI with Andrew Yang if Yang used to be president vs. if he didn’t?
But as for what a president could actually do: create a treaty for countries to sign that ban research into AGI. Very few researchers are aiming for AGI anyway. Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit. I’m not talking about making public speeches on topic; I mean inviting the most important AI researchers to the White House to chat with Stuart Russell and some other folks. There are so many details to work out that we could go back and forth on, but that’s one possibility for something that would be a big deal if it could be made to work.
create a treaty for countries to sign that ban research into AGI.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
Still, I think you are really overestimating the ability of the president to move the scientific community. For instance, we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all. Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism. But the point still stands.
They may not have budged climate scientists, but there other ways they may have influenced policy. Did they (or other partisans) alter the outcomes of Washington Initiative 1631 or 732? That seems hard to evaluate.
Yes, policy can be changed for sure. I was just referring to actually changing minds in the community, as he said—“Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit.”
we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all.
I have updated in your direction.
Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism.
Yep.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
No I meant starting today. My impression is that coalition-building in Washington is tedious work. Scientists agreed to avoid gene editing in humans well before it was possible (I think). In part, that might have made it easier since the distantness of it meant fewer people were researching it to begin with. If AGI is a larger part of an established field, it seems much harder to build a consensus to stop doing it.
FWIW I don’t think that would be a good move. I don’t feel like fully arguing it now, but main points (1) sooner AGI development could well be better despite risk, (2) such restrictions are hard to reverse for a long time after the fact, as the story of human gene editing shows, (3) AGI research is hard to define—arguably, some people are doing it already.
The other thing is that in 20 years, we might want the president on the phone with very specific proposals. What are the odds they’ll spend a weekend discussing AGI with Andrew Yang if Yang used to be president vs. if he didn’t?
But as for what a president could actually do: create a treaty for countries to sign that ban research into AGI. Very few researchers are aiming for AGI anyway. Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit. I’m not talking about making public speeches on topic; I mean inviting the most important AI researchers to the White House to chat with Stuart Russell and some other folks. There are so many details to work out that we could go back and forth on, but that’s one possibility for something that would be a big deal if it could be made to work.
You only mean this as a possibility in the future, if there is any point where AGI is believed to be imminent, right?
Still, I think you are really overestimating the ability of the president to move the scientific community. For instance, we’ve had two presidents now who actively tried to counteract mainstream views on climate-change, and they haven’t budged climate scientists at all. Of course, AI alignment is substantially more scientifically accepted and defensible than climate skepticism. But the point still stands.
They may not have budged climate scientists, but there other ways they may have influenced policy. Did they (or other partisans) alter the outcomes of Washington Initiative 1631 or 732? That seems hard to evaluate.
Yes, policy can be changed for sure. I was just referring to actually changing minds in the community, as he said—“Probably the best starting point would be to get the AI community on board with such a thing. It seems impossible today that consensus could be built about such a thing, but the presidency is a large pulpit.”
I have updated in your direction.
Yep.
No I meant starting today. My impression is that coalition-building in Washington is tedious work. Scientists agreed to avoid gene editing in humans well before it was possible (I think). In part, that might have made it easier since the distantness of it meant fewer people were researching it to begin with. If AGI is a larger part of an established field, it seems much harder to build a consensus to stop doing it.
FWIW I don’t think that would be a good move. I don’t feel like fully arguing it now, but main points (1) sooner AGI development could well be better despite risk, (2) such restrictions are hard to reverse for a long time after the fact, as the story of human gene editing shows, (3) AGI research is hard to define—arguably, some people are doing it already.