“For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy. We report them for completeness, and we will likely discontinue them going forward. Our determination rests primarily on an internal survey of Anthropic staff, in which 0 of 16 participants believed the model could be made into a drop-in replacement for an entry-level researcher with scaffolding and tooling improvements within three months.”
“For ASL-4 evaluations [of CBRN], our automated benchmarks are now largely saturated and no longer provide meaningful signal for rule-out (though as stated above, this is not indicative of harm; it simply means we can no longer rule out certain capabilities that may be pre-requisities to a model having ASL-4 capabilities).”
“We are treating this model as High [for cybersecurity], even though we cannot be certain that it actually has these capabilities, because it meets the requirements of each of our canary thresholds and we therefore cannot rule out the possibility that it is in fact Cyber High.”
Ah yes, this supports my pre-conceived belief that (1) we cannot reliably ascertain whether a model has catastrophically dangerous capabilities, and therefore (2) we need to stop developing increasingly powerful models until we get a handle on things.
The AI Eval Singularity is Near
AI capabilities seem to be doubling every 4-7 months
Humanity’s ability to measure capabilities is growing much more slowly
This implies an “eval singularity”: a point at which capabilities grow faster than our ability to measure them
It seems like the singularity is ~here in cybersecurity, CBRN, and AI R&D (supporting quotes below)
It’s possible that this is temporary, but the people involved seem pretty worried
Appendix—quotes on eval saturation
Opus 4.6
“For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our
automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy. We report them for completeness, and we will likely discontinue them going forward. Our determination rests primarily on an internal survey of Anthropic staff, in which 0 of 16 participants believed the model could be made into a drop-in replacement for an entry-level researcher with scaffolding and tooling improvements within three months.”
“For ASL-4 evaluations [of CBRN], our automated benchmarks are now largely saturated and no longer provide meaningful signal for rule-out (though as stated above, this is not indicative of harm; it simply means we can no longer rule out certain capabilities that may be pre-requisities to a model having ASL-4 capabilities).”
It also saturated ~100% of the cyber evaluations
Codex-5.3
“We are treating this model as High [for cybersecurity], even though we cannot be certain that it actually has these capabilities, because it meets the requirements of each of our canary thresholds and we therefore cannot rule out the possibility that it is in fact Cyber High.”
Ah yes, this supports my pre-conceived belief that (1) we cannot reliably ascertain whether a model has catastrophically dangerous capabilities, and therefore (2) we need to stop developing increasingly powerful models until we get a handle on things.
Related, from an OAI researcher.