Regarding explosive growth in the amount of hardware: I meant to include the scale aspect as well when speaking of a hardware explosion. I tried to outline one of the main reasons I’m skeptical of such an ‘explosion via scaling’ here. In short, in the absence of massive efficiency gains, it seems even less likely that we will see a scale-up explosion in the future.
Incidentally, the graphs you show for the decline in innovations per capita start dropping around 1900 … which is pretty different from the 1960s.
That’s right, but that’s consistent with the per capita drop in innovation being a significant part of the reason why growth rates gradually declined since the 1960s. I didn’t mean to deny that total population size has played a crucial role, as it obviously has and does. But if innovations per capita continue to decline, then even a significant increase in effective population size in the future may not be enough to cause a growth explosion. For example, if the number of employed robots continues to grow at current rates (roughly 12 percent per year), and if future robots eventually come to be the relevant economic population, then declining rates of innovation/economic productivity per capita would mean that the total economic growth rate still doesn’t exceed 12 percent. I realize that you likely expect robot populations to grow much faster in such a future, but I still don’t see what would drive such explosive growth in hardware (even if, in fact especially if, it primarily involves scaling-based growth).
Also, I’m a bit skeptical of the graph showing a 5x drop.
That makes sense.
On the other hand, it’s perhaps worth noting that individual human thinking was increasingly extended by computers after ca. 1950, and yet the rate of innovation per capita still declined. So in that sense, the decline in progress could be seen as being somewhat understated by the graphs, in that the rate of innovation per dollar/scientific instrument/computation/etc. has declined considerably more.
Currently humans are state-of-the-art at various tasks relevant to growth.
We are bottlenecked on scaling up humans by a variety of things (e.g. it takes ~20 years to train up a new human, you can’t invest money into the creation of new humans with the hope of getting a return on it, humans only work ~8 hours a day)
At some point AI / robots will be able to match human performance at these tasks.
AI / robots will not be bottlenecked on those things.
In some sense I agree with you that you have to see efficiency improvements, but the efficiency improvements are things like “you can create new skilled robots in days, compared to the previous SOTA of 20 years”. So I think if you accept (3) then I think you are already accepting massive efficiency improvements.
I don’t see why current robot growth rates are relevant. When you have two different technologies A and B where A works better now, but B is getting better faster than A, then there will predictably be a big jump in the use of B once it exceeds A, and extrapolating the growth rates of B before it exceeds A is going to predictably mislead you.
(For example, I’d guess that in 1975, you would have done better thinking about how / when the personal computer would overtake other office productivity technologies, perhaps based on Moore’s law, rather than trying to extrapolate the growth rate of personal computers. Indeed, according a random website I just found, it looks like the growth rate accelerated till the EDIT: 1980s, though it’s hard to tell from the graph.)
(To be clear, this argument doesn’t necessarily get you to “transformative impact on growth comparable to the industrial revolution”, I’d guess you do need to talk about innovations to get that conclusion. But I’m just not seeing why you don’t expect a ton of scaling even if innovations are rarer, unless you deny (3), but it mostly seems like you don’t deny (3).)
I agree with premise 3. Where I disagree more comes down to the scope of premise 1.
This relates to the diverse class of contributors and bottlenecks to growth under Model 2. So even though it’s true to say that humans are currently “the state-of-the-art at various tasks relevant to growth”, it’s also true to say that computers and robots are currently “the state-of-the-art at various tasks relevant to growth”. Indeed, machines/external tools have been (part of) the state-of-the-art at some tasks for millennia (e.g. in harvesting), and computers and robots in particular have been the state-of-the-art at various tasks relevant to growth for decades (e.g. in technical calculations and manufacturing). And the proportion of tasks at which machines have been driving growth has been gradually increasing (the pictures of Model 2 was an attempt to illustrate this perspective). Yet despite superhuman machines (i.e. machines that are superhuman within specific tasks) playing an increasing role in pushing growth over the past decades, economic growth rates not only failed to increase, but decreased almost by a factor of 2. That is, robots/machines have already been replacing humans at the growth frontier across various tasks in the way described in premises 1-4, yet we still haven’t seen growth increase. So a key question is why we should expect future growth/displacement of this kind to be different. Will it be less gradual? If so, why?
In short, my view is that humans have become an ever smaller part of the combined set of tools pushing growth forward — such that we’re in various senses already a minority force, e.g. in terms of the lifting of heavy objects, performing lengthy math calculations, manufacturing chips, etc. — and I expect this process to gradually continue. I don’t expect a critical point at which growth rates suddenly explode because the machines themselves are already doing such a large share of the heavy lifting, and an increasing proportion of our key bottlenecks to growth are (already) their bottlenecks to faster growth (which must again be distinguished from claims about absolute room for growth; there may be plenty of potential for growth in various domains without there being an extremely fast way to realize that potential).
When you have two different technologies A and B where A works better now, but B is getting better faster than A, then there will predictably be a big jump in the use of B once it exceeds A
Not if B is gradually getting better than A at specific growth-relevant tasks, and if B is getting produced and employed roughly in proportion to how fast it is getting better than A at those specific tasks. In that case, familiar rates of improvement (of B over A) could imply familiar growth rates in the production and employment of B in the future.
But I’m just not seeing why you don’t expect a ton of scaling
Just to be clear, in one sense, I do expect to see a ton of scaling compared to today, I just don’t expect scaling growth rates to explode, such that we see a doubling in a year or faster. In a future robot population that is much larger than the current one, consistent 12 percent annual growth would still amount to producing more robots in a single year than had been produced throughout all history 20 years earlier.
I don’t disagree with any of the above (which is why I emphasized that I don’t think the scaling argument is sufficient to justify a growth explosion). I’m confused why you think the rate of growth of robots is at all relevant, when (general-purpose) robotics seem mostly like a research technology right now. It feels kind of like looking at the current rate of growth of fusion plants as a prediction of the rate of growth of fusion plants after the point where fusion is cheaper than other sources of energy.
(If you were talking about the rate of growth of machines in general I’d find that more relevant.)
By “I am confused by your argument against scaling”, I thought you meant the argument I made here, since that was the main argument I made regarding scaling; the example with robots wasn’t really central.
I’m also a bit confused, because I read your arguments above as being arguments in favor of explosive economic growth rates from hardware scaling and increasing software efficiency. So I’m not sure whether you believe that the factors mentioned in your comment above are sufficient for causing explosive economic growth. Moreover, I don’t yet understand why you believe that hardware scaling would come to grow at much higher rates than it has in the past.
I don’t yet understand why you believe that hardware scaling would come to grow at much higher rates than it has in the past.
If we assume innovations decline, then it is primarily because future AI and robots will be able to automate far more tasks than current AI and robots (and we will get them quickly, not slowly).
Imagine that currently technology A that automates area X gains capabilities at a rate of 5% per year, which ends up leading to a growth rate of 10% per year.
Imagine technology B that also aims to automate area X gains capabilities at a rate of 20% per year, but is currently behind technology A.
Generally, at the point when B exceeds A, I’d expect growth rates of X-automating technologies to grow from 10% to >20% (though not necessarily immediately, it can take time to build the capacity for that growth).
For AI, the area X is “cognitive labor”, technology A is “the current suite of productivity tools”, and technology B is “AI”.
For robots, the area X is “physical labor”, technology A is “classical robotics”, and technology B is “robotics based on foundation models”.
That was just assuming hardware scaling, and it justifies a growth in some particular growth rates, but not a growth explosion. If you add in the software efficiency, then I think you are just straightforwardly generating lots of innovations (what else is leading to the improved software efficiency?) and that’s how you get the growth explosion, at least until you run out of software efficiency improvements to make.
To be clear, I don’t mean to claim that we should give special importance to current growth rates in robotics in particular. I just picked that as an example. But I do think it’s a relevant example, primarily due to the gradual nature of the abilities that robots are surpassing, and the consequent gradual nature of their employment.
Unlike fusion, which is singular in its relevant output (energy), robots produce a diversity of things, and robots cover a wide range of growth-relevant skills that are gradually getting surpassed already. It is this gradual nature of their growth-related abilities that makes them relevant, imo — because they are already doing a lot of work and already contributing a fair deal to the growth we’re currently seeing. (To clarify, I mostly have in mind industrial robots, such as these, the future equivalents of which I also expect to be important to growth; I’d agree that it wouldn’t be so relevant if we were only talking about some prototypes of robots that don’t yet contribute meaningfully to the economy.)
Regarding explosive growth in the amount of hardware: I meant to include the scale aspect as well when speaking of a hardware explosion. I tried to outline one of the main reasons I’m skeptical of such an ‘explosion via scaling’ here. In short, in the absence of massive efficiency gains, it seems even less likely that we will see a scale-up explosion in the future.
That’s right, but that’s consistent with the per capita drop in innovation being a significant part of the reason why growth rates gradually declined since the 1960s. I didn’t mean to deny that total population size has played a crucial role, as it obviously has and does. But if innovations per capita continue to decline, then even a significant increase in effective population size in the future may not be enough to cause a growth explosion. For example, if the number of employed robots continues to grow at current rates (roughly 12 percent per year), and if future robots eventually come to be the relevant economic population, then declining rates of innovation/economic productivity per capita would mean that the total economic growth rate still doesn’t exceed 12 percent. I realize that you likely expect robot populations to grow much faster in such a future, but I still don’t see what would drive such explosive growth in hardware (even if, in fact especially if, it primarily involves scaling-based growth).
That makes sense.
On the other hand, it’s perhaps worth noting that individual human thinking was increasingly extended by computers after ca. 1950, and yet the rate of innovation per capita still declined. So in that sense, the decline in progress could be seen as being somewhat understated by the graphs, in that the rate of innovation per dollar/scientific instrument/computation/etc. has declined considerably more.
I am confused by your argument against scaling.
My understanding of the scale-up argument is:
Currently humans are state-of-the-art at various tasks relevant to growth.
We are bottlenecked on scaling up humans by a variety of things (e.g. it takes ~20 years to train up a new human, you can’t invest money into the creation of new humans with the hope of getting a return on it, humans only work ~8 hours a day)
At some point AI / robots will be able to match human performance at these tasks.
AI / robots will not be bottlenecked on those things.
In some sense I agree with you that you have to see efficiency improvements, but the efficiency improvements are things like “you can create new skilled robots in days, compared to the previous SOTA of 20 years”. So I think if you accept (3) then I think you are already accepting massive efficiency improvements.
I don’t see why current robot growth rates are relevant. When you have two different technologies A and B where A works better now, but B is getting better faster than A, then there will predictably be a big jump in the use of B once it exceeds A, and extrapolating the growth rates of B before it exceeds A is going to predictably mislead you.
(For example, I’d guess that in 1975, you would have done better thinking about how / when the personal computer would overtake other office productivity technologies, perhaps based on Moore’s law, rather than trying to extrapolate the growth rate of personal computers. Indeed, according a random website I just found, it looks like the growth rate accelerated till the EDIT: 1980s, though it’s hard to tell from the graph.)
(To be clear, this argument doesn’t necessarily get you to “transformative impact on growth comparable to the industrial revolution”, I’d guess you do need to talk about innovations to get that conclusion. But I’m just not seeing why you don’t expect a ton of scaling even if innovations are rarer, unless you deny (3), but it mostly seems like you don’t deny (3).)
I agree with premise 3. Where I disagree more comes down to the scope of premise 1.
This relates to the diverse class of contributors and bottlenecks to growth under Model 2. So even though it’s true to say that humans are currently “the state-of-the-art at various tasks relevant to growth”, it’s also true to say that computers and robots are currently “the state-of-the-art at various tasks relevant to growth”. Indeed, machines/external tools have been (part of) the state-of-the-art at some tasks for millennia (e.g. in harvesting), and computers and robots in particular have been the state-of-the-art at various tasks relevant to growth for decades (e.g. in technical calculations and manufacturing). And the proportion of tasks at which machines have been driving growth has been gradually increasing (the pictures of Model 2 was an attempt to illustrate this perspective). Yet despite superhuman machines (i.e. machines that are superhuman within specific tasks) playing an increasing role in pushing growth over the past decades, economic growth rates not only failed to increase, but decreased almost by a factor of 2. That is, robots/machines have already been replacing humans at the growth frontier across various tasks in the way described in premises 1-4, yet we still haven’t seen growth increase. So a key question is why we should expect future growth/displacement of this kind to be different. Will it be less gradual? If so, why?
In short, my view is that humans have become an ever smaller part of the combined set of tools pushing growth forward — such that we’re in various senses already a minority force, e.g. in terms of the lifting of heavy objects, performing lengthy math calculations, manufacturing chips, etc. — and I expect this process to gradually continue. I don’t expect a critical point at which growth rates suddenly explode because the machines themselves are already doing such a large share of the heavy lifting, and an increasing proportion of our key bottlenecks to growth are (already) their bottlenecks to faster growth (which must again be distinguished from claims about absolute room for growth; there may be plenty of potential for growth in various domains without there being an extremely fast way to realize that potential).
Not if B is gradually getting better than A at specific growth-relevant tasks, and if B is getting produced and employed roughly in proportion to how fast it is getting better than A at those specific tasks. In that case, familiar rates of improvement (of B over A) could imply familiar growth rates in the production and employment of B in the future.
Just to be clear, in one sense, I do expect to see a ton of scaling compared to today, I just don’t expect scaling growth rates to explode, such that we see a doubling in a year or faster. In a future robot population that is much larger than the current one, consistent 12 percent annual growth would still amount to producing more robots in a single year than had been produced throughout all history 20 years earlier.
I don’t disagree with any of the above (which is why I emphasized that I don’t think the scaling argument is sufficient to justify a growth explosion). I’m confused why you think the rate of growth of robots is at all relevant, when (general-purpose) robotics seem mostly like a research technology right now. It feels kind of like looking at the current rate of growth of fusion plants as a prediction of the rate of growth of fusion plants after the point where fusion is cheaper than other sources of energy.
(If you were talking about the rate of growth of machines in general I’d find that more relevant.)
By “I am confused by your argument against scaling”, I thought you meant the argument I made here, since that was the main argument I made regarding scaling; the example with robots wasn’t really central.
I’m also a bit confused, because I read your arguments above as being arguments in favor of explosive economic growth rates from hardware scaling and increasing software efficiency. So I’m not sure whether you believe that the factors mentioned in your comment above are sufficient for causing explosive economic growth. Moreover, I don’t yet understand why you believe that hardware scaling would come to grow at much higher rates than it has in the past.
If we assume innovations decline, then it is primarily because future AI and robots will be able to automate far more tasks than current AI and robots (and we will get them quickly, not slowly).
Imagine that currently technology A that automates area X gains capabilities at a rate of 5% per year, which ends up leading to a growth rate of 10% per year.
Imagine technology B that also aims to automate area X gains capabilities at a rate of 20% per year, but is currently behind technology A.
Generally, at the point when B exceeds A, I’d expect growth rates of X-automating technologies to grow from 10% to >20% (though not necessarily immediately, it can take time to build the capacity for that growth).
For AI, the area X is “cognitive labor”, technology A is “the current suite of productivity tools”, and technology B is “AI”.
For robots, the area X is “physical labor”, technology A is “classical robotics”, and technology B is “robotics based on foundation models”.
That was just assuming hardware scaling, and it justifies a growth in some particular growth rates, but not a growth explosion. If you add in the software efficiency, then I think you are just straightforwardly generating lots of innovations (what else is leading to the improved software efficiency?) and that’s how you get the growth explosion, at least until you run out of software efficiency improvements to make.
To be clear, I don’t mean to claim that we should give special importance to current growth rates in robotics in particular. I just picked that as an example. But I do think it’s a relevant example, primarily due to the gradual nature of the abilities that robots are surpassing, and the consequent gradual nature of their employment.
Unlike fusion, which is singular in its relevant output (energy), robots produce a diversity of things, and robots cover a wide range of growth-relevant skills that are gradually getting surpassed already. It is this gradual nature of their growth-related abilities that makes them relevant, imo — because they are already doing a lot of work and already contributing a fair deal to the growth we’re currently seeing. (To clarify, I mostly have in mind industrial robots, such as these, the future equivalents of which I also expect to be important to growth; I’d agree that it wouldn’t be so relevant if we were only talking about some prototypes of robots that don’t yet contribute meaningfully to the economy.)