The first and second examples seems pretty good, and useful reference points.
The third example don’t seem like they are nearly as useful though. What’s particularly unusual about this case is that there are two useful inputs to AI R&D—cognitive labour and compute for experiments—and the former will rise very rapidly but the other will not. In particular, I imagine CS departments also saw compute inputs growing in that time. And I imagine some of the developments discussed (eg proofs about algorithms) only have cognitive labour as an input.
The second example (quant finance), I suppose the ‘data’ input to doing this work stayed constant while the cognitive effort rose. So it works as an example. Though it may be a field with an unusual superabundance of data, unlike ML.
The first example involves a kind of ‘data overhang’ that the cognitive labour quickly eats up. Perhaps in a similar way AGI will “eat up” all the insights that are implicit in existing data from ML experiments.
What i think all the examples currently lack is a measure of how the pace of overall progress changed that isn’t completely made up. Could be interested to list out the achievements in each time period and ask some experts what they think. There an interesting empirical project here I think.
All the examples also lack anything like the scale to which cognitive labour will increase with AGI. This makes comparison even harder. (Though if we can get 3X speed-ups from mild influxes of cognitive labour, that makes 10X speed ups more plausible.)
I tried to edit the paragraph (though LW won’t let me) to:
I think we don’t know what perspective is right, we haven’t had many examples where a huge amount of cognitive labour has been dumped on a scientific field and other inputs to progress have remained constant and we’ve accurately measured how much overall progress in that field accelerates. (Edit: though this comment suggests some interesting examples.)
Thanks, this is a great comment.
The first and second examples seems pretty good, and useful reference points.
The third example don’t seem like they are nearly as useful though. What’s particularly unusual about this case is that there are two useful inputs to AI R&D—cognitive labour and compute for experiments—and the former will rise very rapidly but the other will not. In particular, I imagine CS departments also saw compute inputs growing in that time. And I imagine some of the developments discussed (eg proofs about algorithms) only have cognitive labour as an input.
The second example (quant finance), I suppose the ‘data’ input to doing this work stayed constant while the cognitive effort rose. So it works as an example. Though it may be a field with an unusual superabundance of data, unlike ML.
The first example involves a kind of ‘data overhang’ that the cognitive labour quickly eats up. Perhaps in a similar way AGI will “eat up” all the insights that are implicit in existing data from ML experiments.
What i think all the examples currently lack is a measure of how the pace of overall progress changed that isn’t completely made up. Could be interested to list out the achievements in each time period and ask some experts what they think. There an interesting empirical project here I think.
All the examples also lack anything like the scale to which cognitive labour will increase with AGI. This makes comparison even harder. (Though if we can get 3X speed-ups from mild influxes of cognitive labour, that makes 10X speed ups more plausible.)
I tried to edit the paragraph (though LW won’t let me) to:
I think we don’t know what perspective is right, we haven’t had many examples where a huge amount of cognitive labour has been dumped on a scientific field and other inputs to progress have remained constant and we’ve accurately measured how much overall progress in that field accelerates. (Edit: though this comment suggests some interesting examples.)