“Why aren’t we publicly shaming AI researchers every day? Are we too unwilling to be negative in our pursuit of reducing the chance of doom? Why are we friendly with Anthropic? Anthropic actively accelerates the frontier, currently holds the best coding model, and explicitly aims to build AGI—yet somehow, EAs rally behind them? I’m sure almost everyone agrees that Anthropic could contribute to existential risk, so why do they get a pass? Do we think their AGI is less likely to kill everyone than that of other companies? If so, is this just another utilitarian calculus that we accept even if some worlds lead to EA engineers causing doom themselves? What is going on...”
Preface I have no skin in this game and no inside knowledge, this is just from reading the forums for a few years plus some chats.
I think you’ve put this well. Yes I think many people think Anthropic are more likely to not kill us all than other labs. Which is why you’ll still see their jobs advertised on on the forum and why big EA people like Holden Karnofsky have joined their team.
There are a lot of people that will agree with you that wee should be fighting and shaming not pandering (see pause AI), along with a lot of people who won’t. There’s certainly a (perhaps healthy?) split within the effective altrutism community between those who think we should work technically on the “inside” towards safety and those who think we should just be anti the labs.
Personally I think there’s some “sunk cost ” fallacy here. After Open Phil pumped all that money into open AI, many EAs joined safety teams of labs and there was a huge push towards getting EAs doing technical safety research. After all that it now might feel very hard to turn around now and be anti the labs.
I also think that perhaps the general demeanor of many EAs is bent towards quiet research, policy and technical work rather than protest and loud public criticism, which pushes against that being a core EA contribution to AI safety too.
“Why aren’t we publicly shaming AI researchers every day? Are we too unwilling to be negative in our pursuit of reducing the chance of doom? Why are we friendly with Anthropic? Anthropic actively accelerates the frontier, currently holds the best coding model, and explicitly aims to build AGI—yet somehow, EAs rally behind them? I’m sure almost everyone agrees that Anthropic could contribute to existential risk, so why do they get a pass? Do we think their AGI is less likely to kill everyone than that of other companies? If so, is this just another utilitarian calculus that we accept even if some worlds lead to EA engineers causing doom themselves? What is going on...”
Preface I have no skin in this game and no inside knowledge, this is just from reading the forums for a few years plus some chats.
I think you’ve put this well. Yes I think many people think Anthropic are more likely to not kill us all than other labs. Which is why you’ll still see their jobs advertised on on the forum and why big EA people like Holden Karnofsky have joined their team.
There are a lot of people that will agree with you that wee should be fighting and shaming not pandering (see pause AI), along with a lot of people who won’t. There’s certainly a (perhaps healthy?) split within the effective altrutism community between those who think we should work technically on the “inside” towards safety and those who think we should just be anti the labs.
Personally I think there’s some “sunk cost ” fallacy here. After Open Phil pumped all that money into open AI, many EAs joined safety teams of labs and there was a huge push towards getting EAs doing technical safety research. After all that it now might feel very hard to turn around now and be anti the labs.
I also think that perhaps the general demeanor of many EAs is bent towards quiet research, policy and technical work rather than protest and loud public criticism, which pushes against that being a core EA contribution to AI safety too.