I think it can be a fun exercise is to just interpret CEOs statements literally and see what they imply.
If Dario Amodei claims they donât want to push capabilities, I think an interesting question to ask is in what sense releasing the worldâs best LLM isnât pushing capabilities.
One option that seems possible to me, could be that they no longer consider releasing improved LLMs to meaningfully push the frontier. If Claude-3 spurs OpenAI to push a quicker release of GPT-4.5, this would not be an issue as releasing ever more refined LLMs doesnât meaningfully increase any specific risk.
In that case they would still be true to their word, Anthropic just no longer considers LLMs on their own to be the frontier of capabilities. This seems reasonable to me?
It certainly does seem to push capabilities, although one could argue about whether the extent of it is very significant or not.
Being confused and skeptical about their adherence to their stated philosophy seems justified here, and it is up to them to explain their reasoning behind this decision.
On the margin, this should probably update us towards believing they donât take their stated policy of not advancing the SOTA too seriously.
I think the answer is âyesâ for a general laypersonâs understanding of âpushing capabilitiesâ, but the emerging EA discourse on this seems to be at risk on inflating several questions:
Has Claude-3 shown better capability than other models? Yes under certain specific conditions and benchmarks
Do those benchmarks matter/âactually capture performance of interest? No, in my opinion. Iâd recommend reading Melanie Mitchellâs takes on this.
Does Claude-3â˛s extra capabilities make it more likely to cause an x-risk event? No, or at least the probability that the current frontier AI model will cause an x-risk event has gone from ~epsilon to ~epsilon
Will Claude-3â˛s release increase or decrease x-risk? Very difficult to say, I donât know how people get over cluelessness objections to these questions.
So I guess in your post âfrontierâ is covering 2 separate concepts, the âfrontierâ in terms of published benchmarks and the âfronitierâ in terms of marginal x-risk increase. In my opinion, Claude-3 may be an interesting case where these come apart.
Does Claude-3 push capabilities?
I think it can be a fun exercise is to just interpret CEOs statements literally and see what they imply.
If Dario Amodei claims they donât want to push capabilities, I think an interesting question to ask is in what sense releasing the worldâs best LLM isnât pushing capabilities.
One option that seems possible to me, could be that they no longer consider releasing improved LLMs to meaningfully push the frontier. If Claude-3 spurs OpenAI to push a quicker release of GPT-4.5, this would not be an issue as releasing ever more refined LLMs doesnât meaningfully increase any specific risk.
In that case they would still be true to their word, Anthropic just no longer considers LLMs on their own to be the frontier of capabilities. This seems reasonable to me?
It certainly does seem to push capabilities, although one could argue about whether the extent of it is very significant or not.
Being confused and skeptical about their adherence to their stated philosophy seems justified here, and it is up to them to explain their reasoning behind this decision.
On the margin, this should probably update us towards believing they donât take their stated policy of not advancing the SOTA too seriously.
I think the answer is âyesâ for a general laypersonâs understanding of âpushing capabilitiesâ, but the emerging EA discourse on this seems to be at risk on inflating several questions:
Has Claude-3 shown better capability than other models? Yes under certain specific conditions and benchmarks
Do those benchmarks matter/âactually capture performance of interest? No, in my opinion. Iâd recommend reading Melanie Mitchellâs takes on this.
Does Claude-3â˛s extra capabilities make it more likely to cause an x-risk event? No, or at least the probability that the current frontier AI model will cause an x-risk event has gone from ~epsilon to ~epsilon
Will Claude-3â˛s release increase or decrease x-risk? Very difficult to say, I donât know how people get over cluelessness objections to these questions.
So I guess in your post âfrontierâ is covering 2 separate concepts, the âfrontierâ in terms of published benchmarks and the âfronitierâ in terms of marginal x-risk increase. In my opinion, Claude-3 may be an interesting case where these come apart.