So I’m sympathetic to this perspective, but I want to add a different perspective on this point:
an AI-focused campaign calling out RSPs recently began running ads that single out AI labs for speaking openly about existential risk (quoting leaders acknowledging that things could go catastrophically wrong). I can see why this is a “juicy” lever — most of the public would be pretty astonished/outraged to learn some of the beliefs that are held by AI researchers.
I don’t think they view this as a ‘juicy’ lever, it might just be the right lever (from their PoV)
If some of these leaders/labs think that there is a non-credible chance that AGI could cause an existential risk in the near-term (let’s say 10%+ within ~10/20 years) then I think ‘letting the public know’ has very strong normative and pragmatic support. The astonishment and outrage would rightfully come from the instinctive response of ‘wait, if you believe this, then why the hell are you working on it at all?’
So I guess it’s not just the beliefs the public would find astonishing, but the seeming dissonance between beliefs and actions—and I think that’s a fair response.
I think just letting the public now about AI lab leaders’ p(dooms)s makes sense—in fact, I think most AI researchers are on board with that too (they wouldn’t say these things on podcasts or live on stage if not).
It seems to me this campaign isn’t just meant to raise awareness of X-risk though — it’s meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/legislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labs’ safety agendas might make it less likely they speak about it in the future. It’s kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they don’t should you defect.
So for clarity I’m much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
I’m going to write a response to Connor’s main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update:my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSP’s has come from]
So I’m sympathetic to this perspective, but I want to add a different perspective on this point:
I don’t think they view this as a ‘juicy’ lever, it might just be the right lever (from their PoV)
If some of these leaders/labs think that there is a non-credible chance that AGI could cause an existential risk in the near-term (let’s say 10%+ within ~10/20 years) then I think ‘letting the public know’ has very strong normative and pragmatic support. The astonishment and outrage would rightfully come from the instinctive response of ‘wait, if you believe this, then why the hell are you working on it at all?’
So I guess it’s not just the beliefs the public would find astonishing, but the seeming dissonance between beliefs and actions—and I think that’s a fair response.
I think just letting the public now about AI lab leaders’ p(dooms)s makes sense—in fact, I think most AI researchers are on board with that too (they wouldn’t say these things on podcasts or live on stage if not).
It seems to me this campaign isn’t just meant to raise awareness of X-risk though — it’s meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/legislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labs’ safety agendas might make it less likely they speak about it in the future. It’s kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they don’t should you defect.
So for clarity I’m much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
I’m going to write a response to Connor’s main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update: my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSP’s has come from]
fwiw my opinion is here