Like Akash, I agree with a lot of the object-level points here and disagree with some of the framing / vibes. I’m not sure I can articulate the framing concerns I have, but I do want to say I appreciate you articulating the following points:
Society is waking up to AI risks, and will likely push for a bunch of restrictions on AI progress
Sydney and the ARC Captcha example have made AI safety stuff more salient.
There’s opportunity for substantially more worry about AI risk to emerge after even mild warning events (e.g. AI-powered cyber events, crazier behavior emerging during evals)
Society’s response will be dumb and inefficient in a lot of ways, but could also end up getting pointed in some good directions
The more an org’s AI development / deployment abilities are constrained by safety considerations (whether their own concerns or other stakeholders’), the more safety looks like just another thing you need in order to deploy your powerful AI systems, so that safety work becomes a complement to capabilities work.
Like Akash, I agree with a lot of the object-level points here and disagree with some of the framing / vibes. I’m not sure I can articulate the framing concerns I have, but I do want to say I appreciate you articulating the following points:
Society is waking up to AI risks, and will likely push for a bunch of restrictions on AI progress
Sydney and the ARC Captcha example have made AI safety stuff more salient.
There’s opportunity for substantially more worry about AI risk to emerge after even mild warning events (e.g. AI-powered cyber events, crazier behavior emerging during evals)
Society’s response will be dumb and inefficient in a lot of ways, but could also end up getting pointed in some good directions
The more an org’s AI development / deployment abilities are constrained by safety considerations (whether their own concerns or other stakeholders’), the more safety looks like just another thing you need in order to deploy your powerful AI systems, so that safety work becomes a complement to capabilities work.