On the object level (I made the other comment before reading on), you write:
My impression from talking to Phil Trammell at various times is that it’s just really hard to get such high growth rates from a new technology (and I think he thinks the chance that AGI leads to >20% per year growth rates is lower than I do).
Maybe this is talking about definitions, but I’d say that “like the Industrial Revolution or bigger” doesn’t have to mean literally >20% growth / year. Things could be transformative in others ways, and eventually at least, I feel like things would accelerate almost certainly in a future controlled with or by AGI.
Edit: And I see now that you’re addressing why you feel comfortable disagreeing:
I sort of feel like other people don’t really realise / believe the above so I feel comfortable deviating from them.
I think I might have got the >20% number from Ajeya’s biological anchors report. Of course, I agree that, say, 18% growth might for 20 years might also be at least as big a deal as the Industrial Revolution. It’s just a bit easier to think about a particular growth level (for me anyway). Based on this, maybe I should give some more probability to the “high enough growth for long enough to be at least as big a deal as the Industrial Revolution” than when I was thinking just about the 20% number. (Edit: just to be clear, I did also give some (though not much) probability to non-extreme-economic-growth versions of transformative AI)
I guess this wouldn’t be a big change though so it’s probably(?) not where the disagreement comes from. E.g. if people are counting 10% growth for 10 years as at least as big a deal as the Industrial Revolution I might start thinking that the disagreement mostly comes from definitions.
I phrased my point poorly. I didn’t mean to put the emphasis on the 20% figure, but more on the notion that things will be transformative in a way that fits neatly in the economic growth framework. My concern is that any operationalization of TAI as “x% growth per year(s)” is quite narrow and doesn’t allow for scenarios where AI systems are deployed to secure influence and control over the future first. Maybe there’ll be a war and the “TAI” systems secure influence over the future by wiping out most of the economy except for a few heavily protected compute clusters and resource/production centers. Maybe AI systems are deployed as governance advisors primarily and stay out of the rest of the economy to help with beneficial regulation. And so on.
I think things will almost certainly be transformative one way or another, but if you therefore expect to always see stock market increases of >20%, or increases to other economic growth metrics, then maybe that’s thinking too narrowly. The stock market (or standard indicators of economic growth) are not what ultimately matters. Power-seeking AI systems would prioritize “influence over the long-term future” over “short-term indicators of growth”. Therefore, I’m not sure we see economic growth right when “TAI” arrives. The way I conceptualize “TAI” (and maybe this is different from other operationalizations, though, going by memory, I think it’s compatible with the way Ajeya framed it in her report, since she framed it as “capable of executing a ‘transformative task’”) is that “TAI” is certainly capable of bringing about a radical change in growth mode, eventually, but it may not necessarily be deployed to do that. I think “where’s the point of no return?” is a more important question than “Will AGI systems already transform the economy 1,2,4 years after their invention?”
That said, I don’t think the above difference in how I’d operationalize “TAI” are cruxes between us. From what you say in the writeup, it sounds like you’d be skeptical about both, that AGI systems could transform the economy(/world) directly, and that they could transform it eventually via influence-securing detours.
Thanks, this was interesting. Reading this I think maybe I have a bit of a higher bar than you re what counts as transformative (i.e. at least as big a deal as the industrial revolution). And again, just to say I did give some probability to transformative AI that didn’t act through economic growth. But the main thing that stands out to me is that I haven’t really thought all that much about what the different ways powerful AI might be transformative (as is also the case for almost everything else here too!).
On the object level (I made the other comment before reading on), you write:
Maybe this is talking about definitions, but I’d say that “like the Industrial Revolution or bigger” doesn’t have to mean literally >20% growth / year. Things could be transformative in others ways, and eventually at least, I feel like things would accelerate almost certainly in a future controlled with or by AGI.
Edit: And I see now that you’re addressing why you feel comfortable disagreeing:
I’m not sure about that. :)
I think I might have got the >20% number from Ajeya’s biological anchors report. Of course, I agree that, say, 18% growth might for 20 years might also be at least as big a deal as the Industrial Revolution. It’s just a bit easier to think about a particular growth level (for me anyway). Based on this, maybe I should give some more probability to the “high enough growth for long enough to be at least as big a deal as the Industrial Revolution” than when I was thinking just about the 20% number. (Edit: just to be clear, I did also give some (though not much) probability to non-extreme-economic-growth versions of transformative AI)
I guess this wouldn’t be a big change though so it’s probably(?) not where the disagreement comes from. E.g. if people are counting 10% growth for 10 years as at least as big a deal as the Industrial Revolution I might start thinking that the disagreement mostly comes from definitions.
I phrased my point poorly. I didn’t mean to put the emphasis on the 20% figure, but more on the notion that things will be transformative in a way that fits neatly in the economic growth framework. My concern is that any operationalization of TAI as “x% growth per year(s)” is quite narrow and doesn’t allow for scenarios where AI systems are deployed to secure influence and control over the future first. Maybe there’ll be a war and the “TAI” systems secure influence over the future by wiping out most of the economy except for a few heavily protected compute clusters and resource/production centers. Maybe AI systems are deployed as governance advisors primarily and stay out of the rest of the economy to help with beneficial regulation. And so on.
I think things will almost certainly be transformative one way or another, but if you therefore expect to always see stock market increases of >20%, or increases to other economic growth metrics, then maybe that’s thinking too narrowly. The stock market (or standard indicators of economic growth) are not what ultimately matters. Power-seeking AI systems would prioritize “influence over the long-term future” over “short-term indicators of growth”. Therefore, I’m not sure we see economic growth right when “TAI” arrives. The way I conceptualize “TAI” (and maybe this is different from other operationalizations, though, going by memory, I think it’s compatible with the way Ajeya framed it in her report, since she framed it as “capable of executing a ‘transformative task’”) is that “TAI” is certainly capable of bringing about a radical change in growth mode, eventually, but it may not necessarily be deployed to do that. I think “where’s the point of no return?” is a more important question than “Will AGI systems already transform the economy 1,2,4 years after their invention?”
That said, I don’t think the above difference in how I’d operationalize “TAI” are cruxes between us. From what you say in the writeup, it sounds like you’d be skeptical about both, that AGI systems could transform the economy(/world) directly, and that they could transform it eventually via influence-securing detours.
Thanks, this was interesting. Reading this I think maybe I have a bit of a higher bar than you re what counts as transformative (i.e. at least as big a deal as the industrial revolution). And again, just to say I did give some probability to transformative AI that didn’t act through economic growth. But the main thing that stands out to me is that I haven’t really thought all that much about what the different ways powerful AI might be transformative (as is also the case for almost everything else here too!).