I think just letting the public now about AI lab leaders’ p(dooms)s makes sense—in fact, I think most AI researchers are on board with that too (they wouldn’t say these things on podcasts or live on stage if not).
It seems to me this campaign isn’t just meant to raise awareness of X-risk though — it’s meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/legislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labs’ safety agendas might make it less likely they speak about it in the future. It’s kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they don’t should you defect.
So for clarity I’m much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
I’m going to write a response to Connor’s main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update:my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSP’s has come from]
I think just letting the public now about AI lab leaders’ p(dooms)s makes sense—in fact, I think most AI researchers are on board with that too (they wouldn’t say these things on podcasts or live on stage if not).
It seems to me this campaign isn’t just meant to raise awareness of X-risk though — it’s meant to punish a particular AI lab for releasing what they see as an inadequate safety policy, and to generate public/legislative opposition to that policy.
I think the public should know about X-risk, but I worry using soundbites of it to generate reputatonial harms and counter labs’ safety agendas might make it less likely they speak about it in the future. It’s kind of like a repeated game: if the behavior you want in the coming years is safety-oriented, you should cooperate when your opponent exhibits that behavior. Only when they don’t should you defect.
So for clarity I’m much closer to your position than the ctrl.ai position, and very much agree with your concerns.
But I think, from their perspective, the major AI labs are already defecting by scaling up models that are inherently unsafe despite knowing that this has a significant chance of wiping out humanity (my understanding of ctrl.ai, not my own opinion[1])
I’m going to write a response to Connor’s main post and link to it here that might help explain where their perspective is coming from (based on my own interpretation) [update: my comment is here, which is my attempt to communicate what the ctrl.ai position is, or at least where their scepticism of RSP’s has come from]
fwiw my opinion is here