I feel really confused what the actual right priors here are supposed to be. I find the “but X has fewer parameters” argument only mildly compelling, because I feel like other evidence about similar systems that we’ve observed should easily give us enough evidence to overcome the difference in complexity.
This does mean that a lot of my overall judgement on this question relies on the empirical evidence we have about similar systems, and the concrete gears-level models I have for what has caused growth. AI Impact’s work on discontinuous vs. continuous progress feels somewhat relevant and evidence from other ecological systems also seems reasonably useful.
When I try to understand what exactly happened in terms of growth at a gears-level, I feel like I tend towards more discontinuous hypotheses, because I have a bunch of very concrete, reasonably compelling sounding stories of specific things that caused the relevant shifts, and while I have some gears-level models for what would cause more continuous growth, they feel a lot more nebulous and vague to me, in a way that I think usually doesn’t correspond to truth. The thing that on the margin would feel most compelling to me for the continuous view is something like a concrete zoomed in story of how you get continuous growth from a bunch of humans talking to each other and working with each other over a few generations, that doesn’t immediately abstract things away into high-level concepts like “knowledge” and “capital”.
because I have a bunch of very concrete, reasonably compelling sounding stories of specific things that caused the relevant shifts
Be careful that you don’t have too many stories, or it starts to get continuous again.
More seriously, I don’t know what the small # of factors are for the industrial revolution, and my current sense is that the story can only seem simple for the agricultural revolution because we are so far away and ignoring almost all the details.
It seems like the only factor that looks a priori like it should cause a discontinuity is the transition from hunting+gathering to farming, i.e. if you imagine “total food” as the sum of “food we make” and “food we find” then there could be a discontinuous change in growth rates as “food we make” starts to become large relative to “food we find” (which bounces around randomly but is maybe not really changing). This is blurred because of complementarity between your technology and finding food, but certainly I’m on board with an in-principle argument for a discontinuity as the new mode overtakes the old one.
For the last 10k years my impression is that no one has a very compelling story for discontinuities (put differently: they have waytoo many stories) and it’s mostly a stylized empirical fact that the IR is kind of discontinuous. But I’m provisionally on board with Ben’s basic point that we don’t really have good enough data to know whether growth had been accelerating a bunch in the run-up to the IR.
To the extent things are discontinuous, I’d guess that it’s basically from something similar to the agricultural case—there is continuous growth and random variation, and you see “discontinuities” in the aggregate if a smaller group is significantly outpacing the world, so that by the time they become a large part of the world they are growing significantly faster.
I think this is also reasonably plausible in the AI case (e.g. there is an automated part of the economy doubling every 1-2 years, by the time it gets to be 10% of the economy it’s driving +5%/year growth, 1-2 years later it’s driving +10% growth). But I think quantitatively given the numbers involved and the actual degree of complementarity, this is still unlikely to give you a fast takeoff as I operationalized it. I think if we’re having a serious discussion about “takeoff” that’s probably where the action is, not in any of the kinds of arguments that I dismiss in that post.
I find the “but X has fewer parameters” argument only mildly compelling, because I feel like other evidence about similar systems that we’ve observed should easily give us enough evidence to overcome the difference in complexity.
I mean something much more basic. If you have more parameters then you need to have uncertainty about every parameter. So you can’t just look at how well the best “3 exponentials” hypothesis fits the data, you need to adjust for the fact that this particular “3 exponentials” model has lower prior probability. That is, even if you thought “3 exponentials” was a priori equally likely to a model with fewer parameters, every particular instance of 3 exponentials needs to be less probable than every particular model with fewer parameters.
The thing that on the margin would feel most compelling to me for the continuous view is something like a concrete zoomed in story of how you get continuous growth from a bunch of humans talking to each other and working with each other over a few generations, that doesn’t immediately abstract things away into high-level concepts like “knowledge” and “capital”.
As far as I can tell this is how basically all industries (and scientific domains) work—people learn by doing and talk to each other and they get continuously better, mostly by using and then improving on technologies inherited from other people.
It’s not clear to me whether you are drawing a distinction between modern economic activity and historical cultural accumulation, or whether you feel like you need to see a zoomed-in version of this story for modern economic activity as well, or whether this is a more subtle point about continuous technological progress vs continuous changes in the rate of tech progress, or something else.
I mean something much more basic. If you have more parameters then you need to have uncertainty about every parameter. So you can’t just look at how well the best “3 exponentials” hypothesis fits the data, you need to adjust for the fact that this particular “3 exponentials” model has lower prior probability. That is, even if you thought “3 exponentials” was a priori equally likely to a model with fewer parameters, every particular instance of 3 exponentials needs to be less probable than every particular model with fewer parameters.
Thanks, this was a useful clarification. I agree with this as stated. And I indeed assign substantially more probability to a statement of the form “there were some s-curve like shifts in humanity’s past that made a big difference” than to any specific “these three specific s-curve like shifts are what got us to where we are today”.
As far as I can tell this is how basically all industries (and scientific domains) work—people learn by doing and talk to each other and they get continuously better, mostly by using and then improving on technologies inherited from other people.
It’s not clear to me whether you are drawing a distinction between modern economic activity and historical cultural accumulation, or whether you feel like you need to see a zoomed-in version of this story for modern economic activity as well, or whether this is a more subtle point about continuous technological progress vs continuous changes in the rate of tech progress, or something else.
Hmm, I don’t know, I guess that’s just not really how I would characterize most growth? My model is that most industries start with fast s-curve like growth, then plateau, then often decline. Sure, kind of continuously in the analytical sense, but with large positive and negative changes in the derivative of the growth.
And in my personal experience it’s also less the case that I and the people I work with just get continuously better, it’s more like we kind of flop around until we find something that gets us a lot of traction on something, and then we quickly get much better at the given task, and then we level off again. And it’s pretty easy to get stuck in a rut somewhere and be much less effective than I was years ago, or for an organization to end up in a worse equilibrium and broadly get worse at coordinating, or produce much worse output than previously for other reasons.
Of course enough of those stories could itself give rise to a continuous growth story here, but there is a question here about where the self-similarity lies. Like, many s-curves can also give rise to one big s-curve. Just because I have many s-curve doesn’t mean I get continuous hyperbolic growth. And so seeing lots of relative discontinuous s-curves at the small scale does feel like it’s evidence that we also should expect the macro scale to be a relatively small number of discontinuous s-curves (or more precisely, s-curves whose peak is itself heavy-tail distributed, so that if you run a filter for the s-curves that explain most of the change, you end up with just a few that really mattered).
My model is that most industries start with fast s-curve like growth, then plateau, then often decline
I don’t know exactly what this means, but it seems like most industries in the modern world are characterized by relatively continuous productivity improvements over periods of decades or centuries. The obvious examples to me are semiconductors and AI since I deal most with those. But it also seems true of e.g. manufacturing, agricultural productivity, batteries, construction costs. It seems like industries where the productivity vs time curve is a “fast S-curve” are exceptional, which I assume means we are somehow reading the same data differently. What kind of industries would you characterize this way?
(I agree that e.g. “adoption” is more likely to be an s-curve given that it’s bounded, but productivity seems like the analogy for growth rates.)
it seems like most industries in the modern world are characterized by relatively continuous productivity improvements over periods of decades or centuries
This agrees with my impression. Just in case someone is looking for references for this, see e.g.:
Nagy et al. (2013) - several of the trends they look at, e.g. prices for certain chemical substances, show exponential growth for more than 30 years
Farmer & Lafond (2016) - similar to the previous paper, though fewer trends with data from more than 20 years
Bloom et al. (2020) - reviews trends in research productivity, most of which go back to 1975 and some to 1900
Some early examples from manufacturing (though not covering multiple decades) are reviewed in a famous paper by Arrow (1971), who proposed ‘learning by doing’ as a mechanism.
Note: Actually looking at the graphs in Farmer & Lafond (2016), many of these do sure seem pretty S-curve shaped. As do many of the diagrams in Nagy et al. (2013). I would have to run some real regressions to look at it, but in particular the ones in Farmer & Lafond seem pretty compatible with the basic s-curve model.
Overlapping S-curves are also hard to measure because obviously there are feedback effects between different industries (see my self-similarity comment above). Many of the advances in those fields are driven by exogenous factors, like their inputs getting cheaper, with no substantial improvements in their internal methodologies. One of my models of technological progress (I obviously also share the model of straightforward exponential growth and assign it substantial probability) is that you have nested and overlapping S-curves, which makes it hard to just look at cost/unit output of any individual field.
For analyzing that hypothesis it seems more useful to hold inputs constant and then look at how cost/unit develops, in order to build a model of that isolated chunk of the system (and then obviously also look at the interaction between industries and systems to get a sense of how they interact). But that’s also much harder to do, given that our data is already really messy and noisy.
Thanks for poking at this, it would be quite interesting to me if the “constant exponential growth” story was wrong. Which graphs in Farmer & Lafond (2016) are you referring to? To me, the graph with a summary of all trends only seems to have very few that at first glance look a bit like s-curves. But I agree one would need to go beyond eyeballing to know for sure.
I agree with your other points. My best guess is that input prices and other exogenous factors aren’t that important for some of the trends, e.g. Moore’s Law or agricultural productivity. And I think some of the manufacturing trends in e.g. Arrow (1971) are in terms of output quantity per hour of work rather than prices, and so also seem less dependent on exogenous factors. But I’m more uncertain about this, and agree that in principle dependence on exogenous factors complicates the interpretation.
To me, the graph with a summary of all trends only seems to have very few that at first glance look a bit like s-curves. But I agree one would need to go beyond eyeballing to know for sure.
Yeah, that was the one I was looking at. From very rough eye-balling, it looks like a lot of them have slopes that level off, but obviously super hard to tell just from eye-balling. I might try to find the data and actually check.
I feel really confused what the actual right priors here are supposed to be. I find the “but X has fewer parameters” argument only mildly compelling, because I feel like other evidence about similar systems that we’ve observed should easily give us enough evidence to overcome the difference in complexity.
This does mean that a lot of my overall judgement on this question relies on the empirical evidence we have about similar systems, and the concrete gears-level models I have for what has caused growth. AI Impact’s work on discontinuous vs. continuous progress feels somewhat relevant and evidence from other ecological systems also seems reasonably useful.
When I try to understand what exactly happened in terms of growth at a gears-level, I feel like I tend towards more discontinuous hypotheses, because I have a bunch of very concrete, reasonably compelling sounding stories of specific things that caused the relevant shifts, and while I have some gears-level models for what would cause more continuous growth, they feel a lot more nebulous and vague to me, in a way that I think usually doesn’t correspond to truth. The thing that on the margin would feel most compelling to me for the continuous view is something like a concrete zoomed in story of how you get continuous growth from a bunch of humans talking to each other and working with each other over a few generations, that doesn’t immediately abstract things away into high-level concepts like “knowledge” and “capital”.
Be careful that you don’t have too many stories, or it starts to get continuous again.
More seriously, I don’t know what the small # of factors are for the industrial revolution, and my current sense is that the story can only seem simple for the agricultural revolution because we are so far away and ignoring almost all the details.
It seems like the only factor that looks a priori like it should cause a discontinuity is the transition from hunting+gathering to farming, i.e. if you imagine “total food” as the sum of “food we make” and “food we find” then there could be a discontinuous change in growth rates as “food we make” starts to become large relative to “food we find” (which bounces around randomly but is maybe not really changing). This is blurred because of complementarity between your technology and finding food, but certainly I’m on board with an in-principle argument for a discontinuity as the new mode overtakes the old one.
For the last 10k years my impression is that no one has a very compelling story for discontinuities (put differently: they have way too many stories) and it’s mostly a stylized empirical fact that the IR is kind of discontinuous. But I’m provisionally on board with Ben’s basic point that we don’t really have good enough data to know whether growth had been accelerating a bunch in the run-up to the IR.
To the extent things are discontinuous, I’d guess that it’s basically from something similar to the agricultural case—there is continuous growth and random variation, and you see “discontinuities” in the aggregate if a smaller group is significantly outpacing the world, so that by the time they become a large part of the world they are growing significantly faster.
I think this is also reasonably plausible in the AI case (e.g. there is an automated part of the economy doubling every 1-2 years, by the time it gets to be 10% of the economy it’s driving +5%/year growth, 1-2 years later it’s driving +10% growth). But I think quantitatively given the numbers involved and the actual degree of complementarity, this is still unlikely to give you a fast takeoff as I operationalized it. I think if we’re having a serious discussion about “takeoff” that’s probably where the action is, not in any of the kinds of arguments that I dismiss in that post.
I mean something much more basic. If you have more parameters then you need to have uncertainty about every parameter. So you can’t just look at how well the best “3 exponentials” hypothesis fits the data, you need to adjust for the fact that this particular “3 exponentials” model has lower prior probability. That is, even if you thought “3 exponentials” was a priori equally likely to a model with fewer parameters, every particular instance of 3 exponentials needs to be less probable than every particular model with fewer parameters.
As far as I can tell this is how basically all industries (and scientific domains) work—people learn by doing and talk to each other and they get continuously better, mostly by using and then improving on technologies inherited from other people.
It’s not clear to me whether you are drawing a distinction between modern economic activity and historical cultural accumulation, or whether you feel like you need to see a zoomed-in version of this story for modern economic activity as well, or whether this is a more subtle point about continuous technological progress vs continuous changes in the rate of tech progress, or something else.
Thanks, this was a useful clarification. I agree with this as stated. And I indeed assign substantially more probability to a statement of the form “there were some s-curve like shifts in humanity’s past that made a big difference” than to any specific “these three specific s-curve like shifts are what got us to where we are today”.
Hmm, I don’t know, I guess that’s just not really how I would characterize most growth? My model is that most industries start with fast s-curve like growth, then plateau, then often decline. Sure, kind of continuously in the analytical sense, but with large positive and negative changes in the derivative of the growth.
And in my personal experience it’s also less the case that I and the people I work with just get continuously better, it’s more like we kind of flop around until we find something that gets us a lot of traction on something, and then we quickly get much better at the given task, and then we level off again. And it’s pretty easy to get stuck in a rut somewhere and be much less effective than I was years ago, or for an organization to end up in a worse equilibrium and broadly get worse at coordinating, or produce much worse output than previously for other reasons.
Of course enough of those stories could itself give rise to a continuous growth story here, but there is a question here about where the self-similarity lies. Like, many s-curves can also give rise to one big s-curve. Just because I have many s-curve doesn’t mean I get continuous hyperbolic growth. And so seeing lots of relative discontinuous s-curves at the small scale does feel like it’s evidence that we also should expect the macro scale to be a relatively small number of discontinuous s-curves (or more precisely, s-curves whose peak is itself heavy-tail distributed, so that if you run a filter for the s-curves that explain most of the change, you end up with just a few that really mattered).
I don’t know exactly what this means, but it seems like most industries in the modern world are characterized by relatively continuous productivity improvements over periods of decades or centuries. The obvious examples to me are semiconductors and AI since I deal most with those. But it also seems true of e.g. manufacturing, agricultural productivity, batteries, construction costs. It seems like industries where the productivity vs time curve is a “fast S-curve” are exceptional, which I assume means we are somehow reading the same data differently. What kind of industries would you characterize this way?
(I agree that e.g. “adoption” is more likely to be an s-curve given that it’s bounded, but productivity seems like the analogy for growth rates.)
This agrees with my impression. Just in case someone is looking for references for this, see e.g.:
Nagy et al. (2013) - several of the trends they look at, e.g. prices for certain chemical substances, show exponential growth for more than 30 years
Farmer & Lafond (2016) - similar to the previous paper, though fewer trends with data from more than 20 years
Bloom et al. (2020) - reviews trends in research productivity, most of which go back to 1975 and some to 1900
Some early examples from manufacturing (though not covering multiple decades) are reviewed in a famous paper by Arrow (1971), who proposed ‘learning by doing’ as a mechanism.
Note: Actually looking at the graphs in Farmer & Lafond (2016), many of these do sure seem pretty S-curve shaped. As do many of the diagrams in Nagy et al. (2013). I would have to run some real regressions to look at it, but in particular the ones in Farmer & Lafond seem pretty compatible with the basic s-curve model.
Overlapping S-curves are also hard to measure because obviously there are feedback effects between different industries (see my self-similarity comment above). Many of the advances in those fields are driven by exogenous factors, like their inputs getting cheaper, with no substantial improvements in their internal methodologies. One of my models of technological progress (I obviously also share the model of straightforward exponential growth and assign it substantial probability) is that you have nested and overlapping S-curves, which makes it hard to just look at cost/unit output of any individual field.
For analyzing that hypothesis it seems more useful to hold inputs constant and then look at how cost/unit develops, in order to build a model of that isolated chunk of the system (and then obviously also look at the interaction between industries and systems to get a sense of how they interact). But that’s also much harder to do, given that our data is already really messy and noisy.
Thanks for poking at this, it would be quite interesting to me if the “constant exponential growth” story was wrong. Which graphs in Farmer & Lafond (2016) are you referring to? To me, the graph with a summary of all trends only seems to have very few that at first glance look a bit like s-curves. But I agree one would need to go beyond eyeballing to know for sure.
I agree with your other points. My best guess is that input prices and other exogenous factors aren’t that important for some of the trends, e.g. Moore’s Law or agricultural productivity. And I think some of the manufacturing trends in e.g. Arrow (1971) are in terms of output quantity per hour of work rather than prices, and so also seem less dependent on exogenous factors. But I’m more uncertain about this, and agree that in principle dependence on exogenous factors complicates the interpretation.
Yeah, that was the one I was looking at. From very rough eye-balling, it looks like a lot of them have slopes that level off, but obviously super hard to tell just from eye-balling. I might try to find the data and actually check.