A review of criticisms and an alternative estimate based on the thermodynamic approach
This is a Draft Amnesty Week draft. It may not be polished, up to my usual standards, fully thought through, or fully fact-checked.
This is a Forum post that I wouldn’t have posted without the nudge of Draft Amnesty Week (kudos!). I’d love to see comments and criticisms that take the ideas forward, but it’s unlikely that I will spend much more time on this project.
The following is just the executive summary. The full (draft) report is available here.
This report is the work of Janvi Ahuja and Victoria Schmidt as part of the Epoch FRI Mentorship Programme 2023. We worked on for ~10 hours a week for two months. Tegan McCaslin mentored the project, and Rose Hadshar and Angelina Li provided significant feedback and advice as our peer reviewers.
The Forecasting TAI timelines with biological anchors report produces an estimate of the compute needed to develop a transformative model using 2020 architectures and algorithms. It uses six different biological frameworks to estimate the compute needed to develop a transformative model, one of which is an evolution-based framework. The evolution anchor estimates the amount of computation done by all animals throughout evolution, from the earliest animals with neurons to modern-day humans. In this report, we look into criticisms of the evolution anchor and summarise their effect sizes. We then expand on one such criticism and discuss some of our own criticisms with the biological anchors framework as a whole.
This report may be useful to you if you:
Are interested in biological anchors and defer to the report to determine your AI timelines (and put some weight on the evolutionary anchor). In this case, I would recommend reading the executive summary and reading further on areas of interest. Note, Cotra has posted an update to her original draft and has been interviewed more recently on her timelines.
Are interested in the upper bound estimate of the biological anchors report and want to investigate the most conservative anchor, or otherwise particularly interested in evolutionary anchor. In this case, I might recommend reading the whole report.
Executive summary
Motivation statement
The biological anchors report has influenced many views on when TAI will be developed. A survey by Clarke and McCaffary (March 2023) found it was the second most cited source deferred to on TAI timelines (where the first is “inside view”).
At the outset of the fellowship, our goal was to expand on Nuño Sempere’s criticism pertaining to the cost of simulating the environment, but we found this difficult to make traction on (see footnote here for more information).
Instead, we decided to collate all the criticisms on the evolutionary anchor and their effect sizes.
We also decided to expand on one of the alternative approaches to the evolutionary anchor.
What we did
Summarised critiques of the evolution anchor
Proposed a best guess for an upper bound based on the thermodynamic approach
Proposed reasons we think you should be sceptical of the evolution anchor framework and our results
Suggested how this might affect your TAI timelines
The thermodynamic approach estimates the total amount of energy received by the earth from the Sun and converts this into FLOP. Erdil used the Landauer principle for this conversion, which is the theoretical lower limit of energy consumption of computation. As we expect that most energy was not converted as efficiently as the theoretical lower limit we propose two alternatives:
Using the conversion rate it takes for the brain to convert joules into FLOP
Using the conversion rate it takes for the human body to convert joules into FLOP
As we can expect the average energy-to-information processor to be less efficient than the human brain or body, we expect this is still a conservative upper limit. Our model in the form of a Google sheet is available here.
These approaches result in upper bound estimates which are 4-6 OOMs smaller than the Landauer’s principle approach.
In addition, the upper bounds for both of these estimates are lower than Cotra’s estimate (1E41), at 2E40 for the caloric approach and 6E40 for the brain energy consumption approach. This provides a more informative estimate for the evolution anchor, narrowing down the range to 2E40 FLOP as an upper bound.
Revisiting the Evolution Anchor in the Biological Anchors Report
A review of criticisms and an alternative estimate based on the thermodynamic approach
The following is just the executive summary. The full (draft) report is available here.
This report is the work of Janvi Ahuja and Victoria Schmidt as part of the Epoch FRI Mentorship Programme 2023. We worked on for ~10 hours a week for two months. Tegan McCaslin mentored the project, and Rose Hadshar and Angelina Li provided significant feedback and advice as our peer reviewers.
The Forecasting TAI timelines with biological anchors report produces an estimate of the compute needed to develop a transformative model using 2020 architectures and algorithms. It uses six different biological frameworks to estimate the compute needed to develop a transformative model, one of which is an evolution-based framework. The evolution anchor estimates the amount of computation done by all animals throughout evolution, from the earliest animals with neurons to modern-day humans. In this report, we look into criticisms of the evolution anchor and summarise their effect sizes. We then expand on one such criticism and discuss some of our own criticisms with the biological anchors framework as a whole.
This report may be useful to you if you:
Are interested in biological anchors and defer to the report to determine your AI timelines (and put some weight on the evolutionary anchor). In this case, I would recommend reading the executive summary and reading further on areas of interest. Note, Cotra has posted an update to her original draft and has been interviewed more recently on her timelines.
Are interested in the upper bound estimate of the biological anchors report and want to investigate the most conservative anchor, or otherwise particularly interested in evolutionary anchor. In this case, I might recommend reading the whole report.
Executive summary
Motivation statement
The biological anchors report has influenced many views on when TAI will be developed. A survey by Clarke and McCaffary (March 2023) found it was the second most cited source deferred to on TAI timelines (where the first is “inside view”).
At the outset of the fellowship, our goal was to expand on Nuño Sempere’s criticism pertaining to the cost of simulating the environment, but we found this difficult to make traction on (see footnote here for more information).
Instead, we decided to collate all the criticisms on the evolutionary anchor and their effect sizes.
We also decided to expand on one of the alternative approaches to the evolutionary anchor.
What we did
Summarised critiques of the evolution anchor
Proposed a best guess for an upper bound based on the thermodynamic approach
Proposed reasons we think you should be sceptical of the evolution anchor framework and our results
Suggested how this might affect your TAI timelines
Summarised critiques of the evolution anchor
1E41 FLOP
Environment simulation:
Add costs of simulating an environment and coupling architectures with that environment
Upwards: not quantified
N/A
Environment simulation:
Add environmental simulation cost to the original estimate
+5E27 - ≥4E29 FLOP
1E41 FLOP
Environment simulation:
Simulate whole Earth, molecular simulation
1E60 FLOP
1E60 FLOP
Environment simulation:
Simulate whole Earth, thermodynamic approach
1E45 FLOP
1E45 FLOP
+up to 6 OOM
1E41 − 1E47 FLOP
possibly +>>30 years
N/A
Jennifer Lin
Drastically shortened timelines; not quantified here
Discard biological anchors completely
Upwards; not quantified
N/A
Downwards; not quantified
Decrease the weight of the evolution anchor to 3%
Proposed a best guess for an upper bound based on the thermodynamic approach
The thermodynamic approach estimates the total amount of energy received by the earth from the Sun and converts this into FLOP. Erdil used the Landauer principle for this conversion, which is the theoretical lower limit of energy consumption of computation. As we expect that most energy was not converted as efficiently as the theoretical lower limit we propose two alternatives:
Using the conversion rate it takes for the brain to convert joules into FLOP
Using the conversion rate it takes for the human body to convert joules into FLOP
As we can expect the average energy-to-information processor to be less efficient than the human brain or body, we expect this is still a conservative upper limit. Our model in the form of a Google sheet is available here.
These approaches result in upper bound estimates which are 4-6 OOMs smaller than the Landauer’s principle approach.
In addition, the upper bounds for both of these estimates are lower than Cotra’s estimate (1E41), at 2E40 for the caloric approach and 6E40 for the brain energy consumption approach. This provides a more informative estimate for the evolution anchor, narrowing down the range to 2E40 FLOP as an upper bound.
Proposed reasons we think you should be sceptical of the evolution anchor framework and our results
We expand on fundamental issues with the evolution anchor and how it is derived in Cotra’s report. These include:
Weighing FLOP estimates against developments in compute capacity
Noting that bounding parameter estimates is difficult and sometimes arbitrary
Noting that some of the parameter ranges vary by many orders of magnitude
Noting that FLOP conversion to intelligence is abstract and weird
Suggested how this might affect your TAI timelines
Finally, we provide an overview of what our work could mean for TAI timeline estimates. This is shown below:
If you:
Believe that Cotra’s model and framework for calculating TAI is reasonable
Believe that the thermodynamic estimate for the evolution anchor is better than the brain computation method Cotra uses, and
Believe that our best guess proposal for an improvement upon the thermodynamic estimate is better than the original Landauer approach
You might change your best guess for a FLOP estimate for an upper bound to be 1.79E40 FLOP instead of 1e41 FLOP.
If you:
Believe one of the other criticisms/adjustments noted in section two is legitimate, you might
Use the naive estimated effect sizes to update your estimate of FLOP needed to develop TAI
Choose to investigate it further and update accordingly
Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor
If you:
Believe any of the major reasons to be sceptical listed above you might
Downweight on the evolution anchor and the entire biological anchors report
Consider other ways to assess AI timelines
Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor