Relatedly, I’d be interested to know whether his thoughts on the public’s support for AI pauses or other forms of strict regulation have updated since his last comment exchange with Katja, now that we have many reasonably high-quality polls on the American public’s perception of AI (much more concerned than excited), as well as many more public conversations.
A bit, but more on the willingness of AI experts and some companies to sign the CAIS letter and lend their voices to the view ‘we should go forward very fast with AI, but keep an eye out for better evidence of danger and have the ability to control things later.‘
My model has always been that the public is technophobic, but that ‘this will be constrained like peaceful nuclear power or GMO crops’ isn’t enough to prevent a technology that enables DSA and OOMs (and nuclear power and GMO crops exist, if AGI exists somewhere that place outgrows the rest of the world if the rest of the world sits on the sidelines). If leaders’ understanding of the situation is that public fears are erroneous, and going forward with AI means a hugely better economy (and thus popularity for incumbents) and avoiding a situation where abhorred international rivals can safely disarm their military, then I don’t expect it to be stopped. So the expert views, as defined by who the governments view as experts, are central in my picture.
Visible AI progress like ChatGPT strengthens ‘fear AI disaster’ arguments but at the same time strengthens ‘fear being behind in AI/others having AI’ arguments. The kinds of actions that have been taken so far are mostly of the latter type (export controls, etc), and measures to monitor the situation and perhaps do something later if the evidential situation changes. I.e. they reflect the spirit of the CAIS letter, which companies like OpenAI and such were willing to sign, and not the pause letter which many CAIS letter signatories oppose.
The evals and monitoring agenda is an example of going for value of information rather than banning/stopping AI advances, like I discussed in the comment, and that’s a reason it has had an easier time advancing.
Relatedly, I’d be interested to know whether his thoughts on the public’s support for AI pauses or other forms of strict regulation have updated since his last comment exchange with Katja, now that we have many reasonably high-quality polls on the American public’s perception of AI (much more concerned than excited), as well as many more public conversations.
A bit, but more on the willingness of AI experts and some companies to sign the CAIS letter and lend their voices to the view ‘we should go forward very fast with AI, but keep an eye out for better evidence of danger and have the ability to control things later.‘
My model has always been that the public is technophobic, but that ‘this will be constrained like peaceful nuclear power or GMO crops’ isn’t enough to prevent a technology that enables DSA and OOMs (and nuclear power and GMO crops exist, if AGI exists somewhere that place outgrows the rest of the world if the rest of the world sits on the sidelines). If leaders’ understanding of the situation is that public fears are erroneous, and going forward with AI means a hugely better economy (and thus popularity for incumbents) and avoiding a situation where abhorred international rivals can safely disarm their military, then I don’t expect it to be stopped. So the expert views, as defined by who the governments view as experts, are central in my picture.
Visible AI progress like ChatGPT strengthens ‘fear AI disaster’ arguments but at the same time strengthens ‘fear being behind in AI/others having AI’ arguments. The kinds of actions that have been taken so far are mostly of the latter type (export controls, etc), and measures to monitor the situation and perhaps do something later if the evidential situation changes. I.e. they reflect the spirit of the CAIS letter, which companies like OpenAI and such were willing to sign, and not the pause letter which many CAIS letter signatories oppose.
The evals and monitoring agenda is an example of going for value of information rather than banning/stopping AI advances, like I discussed in the comment, and that’s a reason it has had an easier time advancing.