I haven’t read the report in full (just the short summary) - but I have some initial scepticism, and I’d love to answers to some of the following questions, so I can figure out how much evidence this report is on takeoff speeds. I’ve put the questions roughly in order of subjective importance to my ability to update:
Did you consider Baumol effects, the possibility of technological deflation, and the possibility of technological unemployment, how they affect the profit incentive as tasks are increasingly automated? [My guess is that this effect of all of these is to slow takeoff down, so I’d guess a report that uses simpler models will be noticeably overestimating takeoff speeds.]
How much does this rely on the accuracy of semi-endogenous growth models? Does this model rely on exponential population growth? [I’m asking because as far as I can tell, work relying on semi-endogenous growth models should be pretty weak evidence. First, the “semi” in semi-endogenous growth usually refers to exogenous exponential population growth, which seems unlikely to be a valid assumption. Second, endogenous growth theory has very limited empirical evidence in favour of it (e.g. 1, 2) and I have the impression that this is true for semi-endogenous growth models too. This wouldn’t necessarily be a problem in other fields, but in general I think that economic models with little empirical evidence behind them provide only very weak evidence overall.]
In section 8, the only uncertainty pointing in favour of fast takeoff is “there might be a discontinuous jump in AI capabilities”. Does this mean that, if you don’t think a discontinuous jump in AI capabilities is likely, you should expect slower take-off than your model suggests? How substantial is this effect?
How did you model the AI production function? Relatedly, how did you model constraints like energy costs, data costs, semiconductor costs, silicon costs etc.? [My thoughts: looks like you roughly used a task-based CES model, which seems like a decent choice to me, knowing not much about this! But I’d be curious about the extent to which using this changed your results from Cobb-Douglas.]
I’m vaguely worried that the report proves too much, in that I’d guess that the basic automation of the industrial revolution also automated maybe 70%+ of tasks by pre-industrial revolution GDP. (Of course, generally automation itself wasn’t automated—so I’d be curious on your thoughts about the extent to which this criticism applies at least to the human investment parts of the report.)
That’s all the thoughts that jumped into my head when I read the summary and skimmed the report—sorry if they’re all super obvious if I’d read it more thoroughly! Again, super excited to see models with this level of detail, thanks so much!
The CES task-based model incorporates Baumol effects, in that after AI automates a task the output on that task increases significantly and so its importance to production decreases. The tasks with low output become the bottlenecks to progress.
I’m not sure what exactly you mean by technological deflation. But if AI automates therapy and increases the amount of therapists by 100X then my model won’t imply that the real $ value of therapy industry increases 100X. The price of therapy falls and so there is a more modest increase in the value of therapy.
Re technological unemployment, the model unrealistically assumes that when AI automates (e.g.) 20% of tasks, human workers are immediately reallocated to the remaining 80%. I.e. there is no unemployment until AI automates 100% of tasks. I think this makes sense for things like Copilot that automates/accelerates one part of a job; but is wrong for a hypothetical AI that fully automates a particular job. Modelling delays to reallocating human labour after AI automation would make takeoff slower. My guess is that this will be a bigger deal for the general economy than for AI R&D. Eg maybe AI fully automates the trucking industry, but I don’t expect it to fully automate a particular job within AI R&D. Most of the action with capabilities takeoff speed is with AI R&D (the main effect of AI automation is to accelerate hardware and software progress), so I don’t think modelling this better would affect takeoff speeds by much.
Profit incentives. This is a significant weakness of the report—I don’t explicitly model the incentives faced by firms to invest in AI R&D and do large training runs at all. (More precisely, I don’t endogenise investment decisions as being made to maximise future profits, as happens in some economic models. Epoch is working on a model along these lines.) Instead I assume that once enough significant actors “wake up” to the strategic and economic potential of AI, investments will rise faster than they are today. So one possibility for slower takeoff is that AI firms just really to capture the value they create, and can’t raise the money to go much higher than (e.g.) $5b training runs even after many actors have “woken up”.
I am using semi-endogenous growth models to predict the rate of future software and hardware progress, so they’re very important. I don’t know of a better approach to forecasting how investments in R&D will translate to progress, without investigating the details of where specifically progress might come from (I think that kind of research is very valuable, but it was far beyond the scope of this project). I think semi-endogenous growth models are a better fit to the data than the alternatives (e.g. see this). I do think it’s a valid perspective to say “I just don’t trust any method that tries to predict the rate of technological progress from the amount of R&D investment”, but if you do want to use such a method then I think this is the ~best you can do. In the Monte Carlo analysis, I put large uncertainty bars on the rate of returns to future R&D to represent the fact that the historical relationship between R&D investment and observed progress may fail to hold in the future.
I don’t expect the papers you link to change my mind about this, from reading the abstracts. It seems like your second link is a critique of endogenous growth theory but not semi-endog theory (it says “According to endogenous growth theory, permanent changes in certain policy variables have permanent effects on the rate of economic growth” but this isn’t true of semi-endog theories). It seems like your first link is either looking at ~irrelevant evidence or drawing a the incorrect conclusion (here’s my perspective on the evidence mentioned in its abstract: “the slowing of growth in the OECD countries over the last two decades [Tom: I expect semi-endog theories can explain this better than the neoclassical model. The population growth rate of the scientific workforce as been slowing so we’d expect growth so slow as well; the neoclassical model as (as far as I’m aware) no comparable mechanism for explaining the slowdown.]; the acceleration of growth in several Asian countries since the early 1960s [this is about catch-up growth so wouldn’t expect semi-endog theories to explain it; semi-endog theories are designed to explain growth of the global technological frontier] ; studies of the determinants of growth in a cross-country context [again, semi-endog growth models aren’t designed to explain this kind of thing at all]; and sources of the differences in international productivity levels [again again, semi-endog growth models aren’t designed to explain this kind of thing at all]”.
You could see this as an argument for slower takeoff if you think “I’m pretty sure that looking into the details of where future progress might come from would conclude that progress will be slower than is predicted by the semi-endogenous model”, although this isn’t my current view.
One way to think about this is to start from a method you may trust more that using semi-endog models: just extrapolating past trends in tech progress. But you might worry about this method if you expect R&D inputs to the relevant fields to rise much faster than in recent history (because you expect people to invest more and you expect AI to automate a lot of the work). Naively, your method is going to underestimate the rate of progress. So then using a semi-endog model addresses this problem. It matches the predictions of your initial method when R&D inputs continue to rise at their recent historical rate, but predicts faster progress in scenarios where R&D inputs rise more quickly than in recent history.
> “Does this mean that, if you don’t think a discontinuous jump in AI capabilities is likely, you should expect slower take-off than your model suggests? How substantial is this effect?” The results of the Monte Carlo don’t include any discontinuous jumps (beyond the possibility that there’s a continuous but very-fast transition from “AI that isn’t economically useful” to AGI). So adjusting for discontinuities would only make takeoff faster. My own subjective probabilities do increase the probability of very fast takeoff by 5-10% to account for the possibility of other discontinuities.
“In section 8, the only uncertainty pointing in favour of fast takeoff is “there might be a discontinuous jump in AI capabilities”″ There are other ways that I think my conclusions might be biased in favour of slower takeoff, in particular the ones mentioned here.
“How did you model the AI production function? Relatedly, how did you model constraints like energy costs, data costs, semiconductor costs, silicon costs etc.?”
In the model the capability of the AI trained just depends on the compute used in training and the quality of AI algorithms used; you combine the two multiplicatively. I didn’t model energy/semiconductor/silicon costs except as implicit in FLOP/$ trends); I didn’t model or data costs (which feels like a significant limitation).
The CES task-based model is used as the production function for R&D to improve AI algorithms (“software”) and AI chips (“hardware”), and for GDP. It gives slower takeoff than if you used Cobb Douglas bc you get more bottlenecked by the tasks AI still can’t perform (e.g. tasks done by humans, or tasks done with equipment like experiments).
There’s a parameter rho that controls how close the behaviour is to Cobb Douglas vs a model with very binding bottlenecks. I ultimately settled on a values that make GDP much more bottlenecked by physical infrastructure than R&D progress. This was based on it seeming to me that you could speed up R&D a lot by uploading the smartest minds and running billions of them at 100X speed, but couldn’t increase GDP by nearly as much by having those uploads try to provide people with goods and services (holding the level of technology fixed).
“I’m vaguely worried that the report proves too much, in that I’d guess that the basic automation of the industrial revolution also automated maybe 70%+ of tasks by pre-industrial revolution GDP.” I agree with this! I don’t think it undermines the report—I discuss it here. Interested to hear pushback if you disagree.
This looks really cool, thanks Tom!
I haven’t read the report in full (just the short summary) - but I have some initial scepticism, and I’d love to answers to some of the following questions, so I can figure out how much evidence this report is on takeoff speeds. I’ve put the questions roughly in order of subjective importance to my ability to update:
Did you consider Baumol effects, the possibility of technological deflation, and the possibility of technological unemployment, how they affect the profit incentive as tasks are increasingly automated? [My guess is that this effect of all of these is to slow takeoff down, so I’d guess a report that uses simpler models will be noticeably overestimating takeoff speeds.]
How much does this rely on the accuracy of semi-endogenous growth models? Does this model rely on exponential population growth? [I’m asking because as far as I can tell, work relying on semi-endogenous growth models should be pretty weak evidence. First, the “semi” in semi-endogenous growth usually refers to exogenous exponential population growth, which seems unlikely to be a valid assumption. Second, endogenous growth theory has very limited empirical evidence in favour of it (e.g. 1, 2) and I have the impression that this is true for semi-endogenous growth models too. This wouldn’t necessarily be a problem in other fields, but in general I think that economic models with little empirical evidence behind them provide only very weak evidence overall.]
In section 8, the only uncertainty pointing in favour of fast takeoff is “there might be a discontinuous jump in AI capabilities”. Does this mean that, if you don’t think a discontinuous jump in AI capabilities is likely, you should expect slower take-off than your model suggests? How substantial is this effect?
How did you model the AI production function? Relatedly, how did you model constraints like energy costs, data costs, semiconductor costs, silicon costs etc.? [My thoughts: looks like you roughly used a task-based CES model, which seems like a decent choice to me, knowing not much about this! But I’d be curious about the extent to which using this changed your results from Cobb-Douglas.]
I’m vaguely worried that the report proves too much, in that I’d guess that the basic automation of the industrial revolution also automated maybe 70%+ of tasks by pre-industrial revolution GDP. (Of course, generally automation itself wasn’t automated—so I’d be curious on your thoughts about the extent to which this criticism applies at least to the human investment parts of the report.)
That’s all the thoughts that jumped into my head when I read the summary and skimmed the report—sorry if they’re all super obvious if I’d read it more thoroughly! Again, super excited to see models with this level of detail, thanks so much!
Thanks for these great questions Ben!
To take them point by point:
The CES task-based model incorporates Baumol effects, in that after AI automates a task the output on that task increases significantly and so its importance to production decreases. The tasks with low output become the bottlenecks to progress.
I’m not sure what exactly you mean by technological deflation. But if AI automates therapy and increases the amount of therapists by 100X then my model won’t imply that the real $ value of therapy industry increases 100X. The price of therapy falls and so there is a more modest increase in the value of therapy.
Re technological unemployment, the model unrealistically assumes that when AI automates (e.g.) 20% of tasks, human workers are immediately reallocated to the remaining 80%. I.e. there is no unemployment until AI automates 100% of tasks. I think this makes sense for things like Copilot that automates/accelerates one part of a job; but is wrong for a hypothetical AI that fully automates a particular job. Modelling delays to reallocating human labour after AI automation would make takeoff slower. My guess is that this will be a bigger deal for the general economy than for AI R&D. Eg maybe AI fully automates the trucking industry, but I don’t expect it to fully automate a particular job within AI R&D. Most of the action with capabilities takeoff speed is with AI R&D (the main effect of AI automation is to accelerate hardware and software progress), so I don’t think modelling this better would affect takeoff speeds by much.
Profit incentives. This is a significant weakness of the report—I don’t explicitly model the incentives faced by firms to invest in AI R&D and do large training runs at all. (More precisely, I don’t endogenise investment decisions as being made to maximise future profits, as happens in some economic models. Epoch is working on a model along these lines.) Instead I assume that once enough significant actors “wake up” to the strategic and economic potential of AI, investments will rise faster than they are today. So one possibility for slower takeoff is that AI firms just really to capture the value they create, and can’t raise the money to go much higher than (e.g.) $5b training runs even after many actors have “woken up”.
I am using semi-endogenous growth models to predict the rate of future software and hardware progress, so they’re very important. I don’t know of a better approach to forecasting how investments in R&D will translate to progress, without investigating the details of where specifically progress might come from (I think that kind of research is very valuable, but it was far beyond the scope of this project). I think semi-endogenous growth models are a better fit to the data than the alternatives (e.g. see this). I do think it’s a valid perspective to say “I just don’t trust any method that tries to predict the rate of technological progress from the amount of R&D investment”, but if you do want to use such a method then I think this is the ~best you can do. In the Monte Carlo analysis, I put large uncertainty bars on the rate of returns to future R&D to represent the fact that the historical relationship between R&D investment and observed progress may fail to hold in the future.
I don’t expect the papers you link to change my mind about this, from reading the abstracts. It seems like your second link is a critique of endogenous growth theory but not semi-endog theory (it says “According to endogenous growth theory, permanent changes in certain policy variables have permanent effects on the rate of economic growth” but this isn’t true of semi-endog theories). It seems like your first link is either looking at ~irrelevant evidence or drawing a the incorrect conclusion (here’s my perspective on the evidence mentioned in its abstract: “the slowing of growth in the OECD countries over the last two decades [Tom: I expect semi-endog theories can explain this better than the neoclassical model. The population growth rate of the scientific workforce as been slowing so we’d expect growth so slow as well; the neoclassical model as (as far as I’m aware) no comparable mechanism for explaining the slowdown.] ; the acceleration of growth in several Asian countries since the early 1960s [this is about catch-up growth so wouldn’t expect semi-endog theories to explain it; semi-endog theories are designed to explain growth of the global technological frontier] ; studies of the determinants of growth in a cross-country context [again, semi-endog growth models aren’t designed to explain this kind of thing at all]; and sources of the differences in international productivity levels [again again, semi-endog growth models aren’t designed to explain this kind of thing at all]”.
You could see this as an argument for slower takeoff if you think “I’m pretty sure that looking into the details of where future progress might come from would conclude that progress will be slower than is predicted by the semi-endogenous model”, although this isn’t my current view.
One way to think about this is to start from a method you may trust more that using semi-endog models: just extrapolating past trends in tech progress. But you might worry about this method if you expect R&D inputs to the relevant fields to rise much faster than in recent history (because you expect people to invest more and you expect AI to automate a lot of the work). Naively, your method is going to underestimate the rate of progress. So then using a semi-endog model addresses this problem. It matches the predictions of your initial method when R&D inputs continue to rise at their recent historical rate, but predicts faster progress in scenarios where R&D inputs rise more quickly than in recent history.
> “Does this mean that, if you don’t think a discontinuous jump in AI capabilities is likely, you should expect slower take-off than your model suggests? How substantial is this effect?” The results of the Monte Carlo don’t include any discontinuous jumps (beyond the possibility that there’s a continuous but very-fast transition from “AI that isn’t economically useful” to AGI). So adjusting for discontinuities would only make takeoff faster. My own subjective probabilities do increase the probability of very fast takeoff by 5-10% to account for the possibility of other discontinuities.
“In section 8, the only uncertainty pointing in favour of fast takeoff is “there might be a discontinuous jump in AI capabilities”″ There are other ways that I think my conclusions might be biased in favour of slower takeoff, in particular the ones mentioned here.
“How did you model the AI production function? Relatedly, how did you model constraints like energy costs, data costs, semiconductor costs, silicon costs etc.?”
In the model the capability of the AI trained just depends on the compute used in training and the quality of AI algorithms used; you combine the two multiplicatively. I didn’t model energy/semiconductor/silicon costs except as implicit in FLOP/$ trends); I didn’t model or data costs (which feels like a significant limitation).
The CES task-based model is used as the production function for R&D to improve AI algorithms (“software”) and AI chips (“hardware”), and for GDP. It gives slower takeoff than if you used Cobb Douglas bc you get more bottlenecked by the tasks AI still can’t perform (e.g. tasks done by humans, or tasks done with equipment like experiments).
There’s a parameter rho that controls how close the behaviour is to Cobb Douglas vs a model with very binding bottlenecks. I ultimately settled on a values that make GDP much more bottlenecked by physical infrastructure than R&D progress. This was based on it seeming to me that you could speed up R&D a lot by uploading the smartest minds and running billions of them at 100X speed, but couldn’t increase GDP by nearly as much by having those uploads try to provide people with goods and services (holding the level of technology fixed).
“I’m vaguely worried that the report proves too much, in that I’d guess that the basic automation of the industrial revolution also automated maybe 70%+ of tasks by pre-industrial revolution GDP.” I agree with this! I don’t think it undermines the report—I discuss it here. Interested to hear pushback if you disagree.