Revisiting the Evolution Anchor in the Biological Anchors Report

[Janvi]

A review of criticisms and an alternative estimate based on the thermodynamic approach

This is a Draft Amnesty Week draft. It may not be polished, up to my usual standards, fully thought through, or fully fact-checked.

This is a Forum post that I wouldn’t have posted without the nudge of Draft Amnesty Week (kudos!). I’d love to see comments and criticisms that take the ideas forward, but it’s unlikely that I will spend much more time on this project.

The following is just the executive summary. The full (draft) report is available here.

This report is the work of Janvi Ahuja and Victoria Schmidt as part of the Epoch FRI Mentorship Programme 2023. We worked on for ~10 hours a week for two months. Tegan McCaslin mentored the project, and Rose Hadshar and Angelina Li provided significant feedback and advice as our peer reviewers.

The Forecasting TAI timelines with biological anchors report produces an estimate of the compute needed to develop a transformative model using 2020 architectures and algorithms. It uses six different biological frameworks to estimate the compute needed to develop a transformative model, one of which is an evolution-based framework. The evolution anchor estimates the amount of computation done by all animals throughout evolution, from the earliest animals with neurons to modern-day humans. In this report, we look into criticisms of the evolution anchor and summarise their effect sizes. We then expand on one such criticism and discuss some of our own criticisms with the biological anchors framework as a whole.

This report may be useful to you if you:

• Are interested in biological anchors and defer to the report to determine your AI timelines (and put some weight on the evolutionary anchor). In this case, I would recommend reading the executive summary and reading further on areas of interest. Note, Cotra has posted an update to her original draft and has been interviewed more recently on her timelines.

• Are interested in the upper bound estimate of the biological anchors report and want to investigate the most conservative anchor, or otherwise particularly interested in evolutionary anchor. In this case, I might recommend reading the whole report.

Executive summary

Motivation statement

• The biological anchors report has influenced many views on when TAI will be developed. A survey by Clarke and McCaffary (March 2023) found it was the second most cited source deferred to on TAI timelines (where the first is “inside view”).

• At the outset of the fellowship, our goal was to expand on Nuño Sempere’s criticism pertaining to the cost of simulating the environment, but we found this difficult to make traction on (see footnote here for more information).

• Instead, we decided to collate all the criticisms on the evolutionary anchor and their effect sizes.

• We also decided to expand on one of the alternative approaches to the evolutionary anchor.

What we did

• Summarised critiques of the evolution anchor

• Proposed a best guess for an upper bound based on the thermodynamic approach

• Proposed reasons we think you should be sceptical of the evolution anchor framework and our results

• Suggested how this might affect your TAI timelines

Summarised critiques of the evolution anchor

Critique and approach to incorporate this into the evolution anchor	Expected effect size	Updated evolution anchor	Reference
Original estimate	N/​A	1E41 FLOP	Ajeya Cotra
Environment simulation: Add costs of simulating an environment and coupling architectures with that environment	Upwards: not quantified	N/​A	Jennifer Lin
Environment simulation: Add environmental simulation cost to the original estimate	+5E27 - ≥4E29 FLOP	1E41 FLOP	Nuno Sempere
Environment simulation: Simulate whole Earth, molecular simulation	1E60 FLOP	1E60 FLOP	meanderingmoose
Environment simulation: Simulate whole Earth, thermodynamic approach	1E45 FLOP	1E45 FLOP	Ege Erdil
Anthropic critiques	+up to 6 OOM	1E41 − 1E47 FLOP	Ege Erdil
Paradigm shift	possibly +>>30 years	N/​A	Jennifer Lin
	Drastically shortened timelines; not quantified here	Discard biological anchors completely	Elizier Yudkowsky
Missing architecture search space	Upwards; not quantified	N/​A	Jennifer Lin
Evolutionary algorithms are inefficient	Downwards; not quantified	Decrease the weight of the evolution anchor to 3%	Marius Hobbhahn

Proposed a best guess for an upper bound based on the thermodynamic approach

• The thermodynamic approach estimates the total amount of energy received by the earth from the Sun and converts this into FLOP. Erdil used the Landauer principle for this conversion, which is the theoretical lower limit of energy consumption of computation. As we expect that most energy was not converted as efficiently as the theoretical lower limit we propose two alternatives:

  • Using the conversion rate it takes for the brain to convert joules into FLOP

  • Using the conversion rate it takes for the human body to convert joules into FLOP

• As we can expect the average energy-to-information processor to be less efficient than the human brain or body, we expect this is still a conservative upper limit. Our model in the form of a Google sheet is available here.

• These approaches result in upper bound estimates which are 4-6 OOMs smaller than the Landauer’s principle approach.

• In addition, the upper bounds for both of these estimates are lower than Cotra’s estimate (1E41), at 2E40 for the caloric approach and 6E40 for the brain energy consumption approach. This provides a more informative estimate for the evolution anchor, narrowing down the range to 2E40 FLOP as an upper bound.

Proposed reasons we think you should be sceptical of the evolution anchor framework and our results

• We expand on fundamental issues with the evolution anchor and how it is derived in Cotra’s report. These include:

  • Weighing FLOP estimates against developments in compute capacity

  • Noting that bounding parameter estimates is difficult and sometimes arbitrary

  • Noting that some of the parameter ranges vary by many orders of magnitude

  • Noting that FLOP conversion to intelligence is abstract and weird

Suggested how this might affect your TAI timelines

• Finally, we provide an overview of what our work could mean for TAI timeline estimates. This is shown below:

  • If you:

    • Believe that Cotra’s model and framework for calculating TAI is reasonable

    • Believe that the thermodynamic estimate for the evolution anchor is better than the brain computation method Cotra uses, and

    • Believe that our best guess proposal for an improvement upon the thermodynamic estimate is better than the original Landauer approach

      • You might change your best guess for a FLOP estimate for an upper bound to be 1.79E40 FLOP instead of 1e41 FLOP.

  • If you:

    • Believe one of the other criticisms/​adjustments noted in section two is legitimate, you might

      • Use the naive estimated effect sizes to update your estimate of FLOP needed to develop TAI

      • Choose to investigate it further and update accordingly

      • Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor

  • If you:

    • Believe any of the major reasons to be sceptical listed above you might

      • Downweight on the evolution anchor and the entire biological anchors report

      • Consider other ways to assess AI timelines

      • Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor

[OscarD🔸]

Thanks for writing this! I agree that bioanchors is still worth engaging with and revisiting given how important it has been and is.

I like the overall approach of trying to quantify how much different criticism would update the 1e41 estimate. I don’t feel well-placed to comment on the thermodynamic approach part, but if it works roughly as you outline this seems like an important robustness check for the evolution anchor.

I left a bunch of more minor comments in the report.