Tom Davidson’s model is often referred to in the Community, but it is entirely reliant on the current paradigm + scale reaching AGI.
This seems wrong.
It does use constants from the historical deep learning field to provide guesses for parameters and it assumes that compute is an important driver of AI progress.
These are much weaker assumptions than you seem to be implying.
Note also that this work is based on earlier work like bio anchors which was done just as the current paradigm and scaling were being established. (It was published in the same year as Kaplan et al.)
In this framework, AGI is developed by improving and scaling up approaches within the current ML paradigm, not by discovering new algorithmic paradigms.
So the kinda zoomed out idea behind the Compute-centric framwork is that I’m assuming something like the current paradigm is going to lead to human-level AI and further, and I’m assuming that we get there by scaling up and improving the current algorithmic approaches. So it’s going to look like better versions of transformers that are more efficient and that allow for larger context windows...”
Both of these seem to be pretty scaling-maximalist to me, so I don’t think the quote seems wrong, at least to me? It’d be pretty hard to make a model which includes the possibility of the paradigm not getting us to AGI and then needing a period of exploration across the field to find the other breakthroughs needed.
This seems wrong.
It does use constants from the historical deep learning field to provide guesses for parameters and it assumes that compute is an important driver of AI progress.
These are much weaker assumptions than you seem to be implying.
Note also that this work is based on earlier work like bio anchors which was done just as the current paradigm and scaling were being established. (It was published in the same year as Kaplan et al.)
From the summary page on Open Phil:
From this presentation about it to GovAI (from April 2023) at 05:10:
Both of these seem to be pretty scaling-maximalist to me, so I don’t think the quote seems wrong, at least to me? It’d be pretty hard to make a model which includes the possibility of the paradigm not getting us to AGI and then needing a period of exploration across the field to find the other breakthroughs needed.