A few points on the Bio Anchors framework, and why I expect TAI to require much more compute than used by the human brain:
1. Today we routinely use computers with as much compute as the human brain. Joe Carlsmith’s OpenPhil report finds the brain uses between 10^13 and 10^17 FLOP/s. He points out that Nvidia’s V100 GPU retailing for $10,000 currently performs 10^14 FLOP/s.
2. Ajeya Cotra’s Bio Anchors report shows that AlphaStar’s training run used 10^23 FLOP, the equivalent of running a human brain-sized computer with 10^15 FLOP/s for four years. The Human Lifetime anchor therefore estimates that a transformative model could already be trained with today’s levels of compute with 22% probability, but we have not seen such a model so far.
Why, then, do we not have transformative AI? Maybe it’s right around the corner, with the human lifetime anchor estimating a 50% chance of transformative AI by 2032. I’m more inclined to say that this reduces my credence in the report’s short timelines based on the compute of the human brain. The Evolution anchor seems to me like a more realistic prediction, with 50% probability of TAI by 2090.
I’d also like to see more research on the evolution anchor. The Evolution anchor is the part of the report that Ajeya says she “spent the least amount of time thinking about.” Its estimates of the size of evolutionary history are primarily from this 2009 blog post, and its final calculation assumes that all of our ancestors had brains the size of nematodes and that the organism population of the Earth has been constant for 1 billion years. These are extremely rough assumptions, and Ajeya also says that “there are plausible arguments that I have underestimated true evolutionary computation here in ways that would be somewhat time-consuming to correct.” On the other hand, it seems reasonable to me that our scientists could generate algorithmic improvements much faster than evolution did, though Ajeya notes that “some ML researchers would want to argue that we would need substantially more computation than was performed in the brains of all animals over evolutionary history; while I disagree with this, it seems that the Evolution Anchor hypothesis should place substantial weight on this possibility.”
This (pop science) article provides two interesting critiques of the analogy between the human brain and neural nets.
“Neural nets are typically trained by “supervised learning”. This is very different from how humans typically learn. Most human learning is “unsupervised”, which means we’re not explicitly told what the “right” response is for a given stimulus. We have to work this out ourselves.”
“Another difference is the sheer scale of data used to train AI. The GPT-3 model was trained on 400 billion words, mostly taken from the internet. At a rate of 150 words per minute, it would take a human nearly 4,000 years to read this much text.”
I’m not sure the direct implication for timelines here. You might be able to argue that these disanalogies mean that neural nets will require less compute than the brain. But an interesting point of disanalogy, to correct any misconceptions that neural networks are “just like the brain”.
Concerns with BioAnchors Timelines
A few points on the Bio Anchors framework, and why I expect TAI to require much more compute than used by the human brain:
1. Today we routinely use computers with as much compute as the human brain. Joe Carlsmith’s OpenPhil report finds the brain uses between 10^13 and 10^17 FLOP/s. He points out that Nvidia’s V100 GPU retailing for $10,000 currently performs 10^14 FLOP/s.
2. Ajeya Cotra’s Bio Anchors report shows that AlphaStar’s training run used 10^23 FLOP, the equivalent of running a human brain-sized computer with 10^15 FLOP/s for four years. The Human Lifetime anchor therefore estimates that a transformative model could already be trained with today’s levels of compute with 22% probability, but we have not seen such a model so far.
Why, then, do we not have transformative AI? Maybe it’s right around the corner, with the human lifetime anchor estimating a 50% chance of transformative AI by 2032. I’m more inclined to say that this reduces my credence in the report’s short timelines based on the compute of the human brain. The Evolution anchor seems to me like a more realistic prediction, with 50% probability of TAI by 2090.
I’d also like to see more research on the evolution anchor. The Evolution anchor is the part of the report that Ajeya says she “spent the least amount of time thinking about.” Its estimates of the size of evolutionary history are primarily from this 2009 blog post, and its final calculation assumes that all of our ancestors had brains the size of nematodes and that the organism population of the Earth has been constant for 1 billion years. These are extremely rough assumptions, and Ajeya also says that “there are plausible arguments that I have underestimated true evolutionary computation here in ways that would be somewhat time-consuming to correct.” On the other hand, it seems reasonable to me that our scientists could generate algorithmic improvements much faster than evolution did, though Ajeya notes that “some ML researchers would want to argue that we would need substantially more computation than was performed in the brains of all animals over evolutionary history; while I disagree with this, it seems that the Evolution Anchor hypothesis should place substantial weight on this possibility.”
This (pop science) article provides two interesting critiques of the analogy between the human brain and neural nets.
“Neural nets are typically trained by “supervised learning”. This is very different from how humans typically learn. Most human learning is “unsupervised”, which means we’re not explicitly told what the “right” response is for a given stimulus. We have to work this out ourselves.”
“Another difference is the sheer scale of data used to train AI. The GPT-3 model was trained on 400 billion words, mostly taken from the internet. At a rate of 150 words per minute, it would take a human nearly 4,000 years to read this much text.”
I’m not sure the direct implication for timelines here. You might be able to argue that these disanalogies mean that neural nets will require less compute than the brain. But an interesting point of disanalogy, to correct any misconceptions that neural networks are “just like the brain”.