Yeah, with the word “capability” I meant completely new capabilities (in Aschenbrenner’s case, the relevant new capabilities would be general intelligence abilities such as the learning-planning ability), but I can see that for example object permanence could be called a new capability. I maybe should have used a better word there. Basically, my argument is that while the image generators have become better at generating images, they haven’t gotten anything that would take them nearer towards AGI.
I’ll grant you, as does he, that unhobbling is hand-wavy and hard to measure (although that by no means implies it isn’t real).
I’m not claiming that unhobbling isn’t real, and I think that the mentioned improvements such as CoT and scaffolding etc. really do make models better. But do they make them exponentially better? Can we expect the increases to continue exponentially in the future? I’m going to say no. So I think it’s unsubstantiated to measure them with orders of magnitude.
But we can certainly measure floating point operations! So accusing him of using “OOMs” as a unit, and one that is unmeasurable/detached from reality, surprises me.
Most of the time, when he says “OOM”, he doesn’t refer to FLOPs, he refers to the abstract OOMs that somehow encompass all three axes he mentioned. So while some of it is measurable, as a whole it is not.
It seems to me that you are missing my point. I’m not trying to dismiss or debunk Aschenbrenner. My point is to call out that what he is doing is harmful to everyone, including those who believe AGI is imminent.
If you believe that AGI is coming soon, then shouldn’t you try to convince other people of this? If so, shouldn’t you be worried that people like Aschenbrenner ruin that by presenting themselves like conspiracy theorists?
Yes! That is why what Aschenbrenner is doing is so harmful, he is using an emotional or narrative argument instead of a real object-level argument. Like you say, we need to do better.
I have read the technical claims made by Aschenbrenner and many other AI optimists, and I’m not convinced. There is no evidence for any kind of general intelligence abilities surfacing in any of the current AI systems. People have been trying to do that for decades, and for the part couple of years, but there has been almost no progress on that front at all (in-context learning is one of the biggest ones I can think, and it can hardly even be called learning). While I do think that some action can be taken, what Aschenbrenner suggests is, as I iterate in my text, too much given our current evidence. Extraordinary claims require extraordinary evidence, as it is said.