I’m worried that trying to estimate σ by looking at wages is subject to lots of noise due to assumptions being violated, which could result in the large discrepancy you see between the two estimates.
One worry: I would guess that Anthropic could derive more output from extra researchers (1.5x/doubling?) than from extra GPUs (1.18x/doubling?), yet it spends more on compute than researchers. In particular I’d guess alpha/beta = 2.5, and wages/r_{research} is around 0.28 (maybe you have better data here). Under Cobb-Douglas and perfect competition these should be equal, but they’re off by a factor of 9! I’m not totally sure but I think this would give you strange parameter values in CES as well. This huge gap between output elasticity and where firms are spending their money is strange to me, so I strongly suspect that one of the assumptions is broken rather than σ just being some extreme value like −0.10 or 2.58 with large firm fixed effects.
My guess at why: The AI industry is very different than it was in 2012 so it is plausible these firm fixed effects have actually greatly changed over time, which would affect the regression coefficients. Just some examples of possible changes over time:
Some of the compute growth over time is due to partnerships, e.g. Amazon’s $4 billion investment in Anthropic. Maybe these don’t reflect buying compute at market prices.
The value of equity now is hundreds of times higher than in 2017, and employees are compensated mainly in equity.
The cost of hiring might be primarily in onboarding / management capacity rather than wages; this would hit OpenAI and Anthropic harder since 2021 since they’ve both grown furiously in that period.
The whole industry is much larger now and elasticity of substitution might not be constant; if so this is worrying because to predict whether there’s a software-only singularity we’ll need to extrapolate over more orders of magnitude of growth and the human labor → AI labor transition.
Companies could be investing in compute because it increases their revenue (since ChatGPT), or stockpiling compute so they can take advantage of research automation later.
Nevertheless I’m excited about the prospect of estimating σ and ϕ and I’m glad this was posted. Are you planning follow-up work, or is there other economic data we could theoretically collect that could give us higher confidence estimates?
I take your overall point as the static optimization problem may not be properly specified. For example, costs may not be linear in labor size because of adjustment costs to growing very quickly or costs may not be linear in compute because of bulk discounting. Moreover, these non-linear costs may be changing over time (e.g., adjustment costs might only matter in 2021-2024 as OpenAI, Anthropic have been scaling labor aggressively). I agree that this would bias the estimate of σ. Given the data we have, there should be some way to at least partially deal with this (e.g., by adding lagged labor as a control). I’ll have to think about it more.
On some of the smaller comments:
wages/r_{research} is around 0.28 (maybe you have better data here)
The best data we have is The Information’s article that OpenAI spent $700M on salaries and $1000M on research compute in 2024, so the wLrK=.7 (assuming you meant wLrK instead of wr).
The whole industry is much larger now and elasticity of substitution might not be constant; if so this is worrying because to predict whether there’s a software-only singularity we’ll need to extrapolate over more orders of magnitude of growth and the human labor → AI labor transition.
I agree.σ might not be constant over time, which is a problem for both estimation/extrapolation and also predicting what an intelligence explosion might look like. For example, if σ falls over time, then we may have a foom for a bit until σ falls below 1 and then fizzles. I’ve been thinking about writing something up about this.
Are you planning follow-up work, or is there other economic data we could theoretically collect that could give us higher confidence estimates?
Yes, although we’re not decided yet on what is the most useful to follow-up on. Very short-term there is trying to accomodate non-linear pricing. Of course, data on what non-linear pricing looks like would be helpful e.g., how does Nvidia bulk discount.
We also may try to estimate ϕ with the data we have.
I’m worried that trying to estimate σ by looking at wages is subject to lots of noise due to assumptions being violated, which could result in the large discrepancy you see between the two estimates.
One worry: I would guess that Anthropic could derive more output from extra researchers (1.5x/doubling?) than from extra GPUs (1.18x/doubling?), yet it spends more on compute than researchers. In particular I’d guess alpha/beta = 2.5, and wages/r_{research} is around 0.28 (maybe you have better data here). Under Cobb-Douglas and perfect competition these should be equal, but they’re off by a factor of 9! I’m not totally sure but I think this would give you strange parameter values in CES as well. This huge gap between output elasticity and where firms are spending their money is strange to me, so I strongly suspect that one of the assumptions is broken rather than σ just being some extreme value like −0.10 or 2.58 with large firm fixed effects.
My guess at why: The AI industry is very different than it was in 2012 so it is plausible these firm fixed effects have actually greatly changed over time, which would affect the regression coefficients. Just some examples of possible changes over time:
Some of the compute growth over time is due to partnerships, e.g. Amazon’s $4 billion investment in Anthropic. Maybe these don’t reflect buying compute at market prices.
The value of equity now is hundreds of times higher than in 2017, and employees are compensated mainly in equity.
The cost of hiring might be primarily in onboarding / management capacity rather than wages; this would hit OpenAI and Anthropic harder since 2021 since they’ve both grown furiously in that period.
The whole industry is much larger now and elasticity of substitution might not be constant; if so this is worrying because to predict whether there’s a software-only singularity we’ll need to extrapolate over more orders of magnitude of growth and the human labor → AI labor transition.
Companies could be investing in compute because it increases their revenue (since ChatGPT), or stockpiling compute so they can take advantage of research automation later.
Nevertheless I’m excited about the prospect of estimating σ and ϕ and I’m glad this was posted. Are you planning follow-up work, or is there other economic data we could theoretically collect that could give us higher confidence estimates?
Thanks for the insightful comment.
I take your overall point as the static optimization problem may not be properly specified. For example, costs may not be linear in labor size because of adjustment costs to growing very quickly or costs may not be linear in compute because of bulk discounting. Moreover, these non-linear costs may be changing over time (e.g., adjustment costs might only matter in 2021-2024 as OpenAI, Anthropic have been scaling labor aggressively). I agree that this would bias the estimate of σ. Given the data we have, there should be some way to at least partially deal with this (e.g., by adding lagged labor as a control). I’ll have to think about it more.
On some of the smaller comments:
The best data we have is The Information’s article that OpenAI spent $700M on salaries and $1000M on research compute in 2024, so the wLrK=.7 (assuming you meant wLrK instead of wr).
I agree.σ might not be constant over time, which is a problem for both estimation/extrapolation and also predicting what an intelligence explosion might look like. For example, if σ falls over time, then we may have a foom for a bit until σ falls below 1 and then fizzles. I’ve been thinking about writing something up about this.
Yes, although we’re not decided yet on what is the most useful to follow-up on. Very short-term there is trying to accomodate non-linear pricing. Of course, data on what non-linear pricing looks like would be helpful e.g., how does Nvidia bulk discount.
We also may try to estimate ϕ with the data we have.