So Iām sympathetic to this perspective, but I want to add a different perspective on this point:
an AI-focused campaign calling out RSPs recently began running ads that single out AI labs for speaking openly about existential risk (quoting leaders acknowledging that things could go catastrophically wrong). I can see why this is a ājuicyā lever ā most of the public would be pretty astonished/āoutraged to learn some of the beliefs that are held by AI researchers.
I donāt think they view this as a ājuicyā lever, it might just be the right lever (from their PoV)
If some of these leaders/ālabs think that there is a non-credible chance that AGI could cause an existential risk in the near-term (letās say 10%+ within ~10/ā20 years) then I think āletting the public knowā has very strong normative and pragmatic support. The astonishment and outrage would rightfully come from the instinctive response of āwait, if you believe this, then why the hell are you working on it at all?ā
So I guess itās not just the beliefs the public would find astonishing, but the seeming dissonance between beliefs and actionsāand I think thatās a fair response.
I think just letting the public now about AI lab leadersā p(dooms)s makes senseāin fact, I think most AI researchers are on board with that too (they wouldnāt say these things on podcasts or live on stage if not).
It seems to me this campaign isnāt just meant to raise awareness of X-risk though ā itās meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/ālegislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labsā safety agendas might make it less likely they speak about it in the future. Itās kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they donāt should you defect.
So for clarity Iām much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
Iām going to write a response to Connorās main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update:my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSPās has come from]
So Iām sympathetic to this perspective, but I want to add a different perspective on this point:
I donāt think they view this as a ājuicyā lever, it might just be the right lever (from their PoV)
If some of these leaders/ālabs think that there is a non-credible chance that AGI could cause an existential risk in the near-term (letās say 10%+ within ~10/ā20 years) then I think āletting the public knowā has very strong normative and pragmatic support. The astonishment and outrage would rightfully come from the instinctive response of āwait, if you believe this, then why the hell are you working on it at all?ā
So I guess itās not just the beliefs the public would find astonishing, but the seeming dissonance between beliefs and actionsāand I think thatās a fair response.
I think just letting the public now about AI lab leadersā p(dooms)s makes senseāin fact, I think most AI researchers are on board with that too (they wouldnāt say these things on podcasts or live on stage if not).
It seems to me this campaign isnāt just meant to raise awareness of X-risk though ā itās meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/ālegislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labsā safety agendas might make it less likely they speak about it in the future. Itās kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they donāt should you defect.
So for clarity Iām much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
Iām going to write a response to Connorās main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update: my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSPās has come from]
fwiw my opinion is here