On “transformative AI”: I agree that this is quite vague and not as well-defined as it would ideally be, and is not the kind of thing I think we could just hand to superforecasters. But I think it is pointing at something important that I haven’t seen a better way of pointing at.
I like the definition given in Bio Anchors (which you link to), which includes a footnote addressing the fact that AI could be transformative without literally causing GDP growth to behave as described. I’m sure there are imperfections remaining, and it remains vague, but I think most people can get a pretty good idea of what’s being pointed at there, and I think it reasonably fleshes out the vaguer, simpler definition (which I think is also useful for giving a high-level impression).
In this series, I mostly stuck with the simple definition because I think the discussion of PASTA and digital people makes it fairly easy to see what kind of specific thing I’m pointing at, in a different way.
I am not aware of places where it’s implied that “transformative AI” is a highly well-defined concept suitable for superforecasters (and I don’t think the example you gave in fact implies this), but I’m happy to try to address them if you point them out.
On PASTA: my view is that there is a degree of automation that would in fact result in dramatically faster scientific progress than we’ve ever seen before. I don’t think this is self-evident, or tightly proven by the series, but it is something I believe, and I think the series does a reasonable job pointing to the main intuitions behind why I believe it (in particular, the theoretical feedback loop this would create, the “modeling the human trajectory” projection of what we might expect if the “population bottleneck” were removed, and the enormous transformative potential of particular technologies that might result).
I’m sure there are imperfections remaining, and it remains vague, but I think most people can get a pretty good idea of what’s being pointed at there, and I think it reasonably fleshes out the vaguer, simpler definition (which I think is also useful for giving a high-level impression).
I’d disagree that most people can get a good idea of what’s being pointed at; not least for the reasons I outlined in Section 1.2 above, regarding how advanced software could already reasonably be claimed to have “precipitate[d] a transition comparable to (or more significant than) the agricultural or industrial revolution”. :)
So I’d also disagree that it “reasonably fleshes out the vaguer, simpler definition”. Indeed, I don’t think “Transformative AI” is a much clearer term than, say, “Advanced AI” or “Powerful AI”, but it often seems used as though it’s much clearer (see e.g. below).
I am not aware of places where it’s implied that “transformative AI” is a highly well-defined concept suitable for superforecasters (and I don’t think the example you gave in fact implies this), but I’m happy to try to address them if you point them out.
My point wasn’t about superforecasters in particular. Rather, my point was that the current definitions of TAI are so vague that it doesn’t make much sense to talk about, say, “the year by which transformative AI will be developed”. Again, it is highly unclear what would count and how one would resolve any forecast about it.
As for (super)forecasters, I wonder: if the concept/definition is not “suitable for superforecasters” — that is, for clearly resolvable forecasts — why is it suitable for attempts to forecast this “one number[i.e. the year by which transformative AI will be developed]”? If one doesn’t think it allows for clearly resolvable forecasts, perhaps it would be good to note that from the outset, and when making estimates such as “more than a 10% chance of ‘transformative AI’ within 15 years”.
sidenote: There has been an argument that ‘radically transformative AI’ is a better term for the Industrial Revolution definition, given the semantic bleaching already taking place with ‘transformative AI’.
On “transformative AI”: I agree that this is quite vague and not as well-defined as it would ideally be, and is not the kind of thing I think we could just hand to superforecasters. But I think it is pointing at something important that I haven’t seen a better way of pointing at.
I like the definition given in Bio Anchors (which you link to), which includes a footnote addressing the fact that AI could be transformative without literally causing GDP growth to behave as described. I’m sure there are imperfections remaining, and it remains vague, but I think most people can get a pretty good idea of what’s being pointed at there, and I think it reasonably fleshes out the vaguer, simpler definition (which I think is also useful for giving a high-level impression).
In this series, I mostly stuck with the simple definition because I think the discussion of PASTA and digital people makes it fairly easy to see what kind of specific thing I’m pointing at, in a different way.
I am not aware of places where it’s implied that “transformative AI” is a highly well-defined concept suitable for superforecasters (and I don’t think the example you gave in fact implies this), but I’m happy to try to address them if you point them out.
On PASTA: my view is that there is a degree of automation that would in fact result in dramatically faster scientific progress than we’ve ever seen before. I don’t think this is self-evident, or tightly proven by the series, but it is something I believe, and I think the series does a reasonable job pointing to the main intuitions behind why I believe it (in particular, the theoretical feedback loop this would create, the “modeling the human trajectory” projection of what we might expect if the “population bottleneck” were removed, and the enormous transformative potential of particular technologies that might result).
I’d disagree that most people can get a good idea of what’s being pointed at; not least for the reasons I outlined in Section 1.2 above, regarding how advanced software could already reasonably be claimed to have “precipitate[d] a transition comparable to (or more significant than) the agricultural or industrial revolution”. :)
So I’d also disagree that it “reasonably fleshes out the vaguer, simpler definition”. Indeed, I don’t think “Transformative AI” is a much clearer term than, say, “Advanced AI” or “Powerful AI”, but it often seems used as though it’s much clearer (see e.g. below).
My point wasn’t about superforecasters in particular. Rather, my point was that the current definitions of TAI are so vague that it doesn’t make much sense to talk about, say, “the year by which transformative AI will be developed”. Again, it is highly unclear what would count and how one would resolve any forecast about it.
As for (super)forecasters, I wonder: if the concept/definition is not “suitable for superforecasters” — that is, for clearly resolvable forecasts — why is it suitable for attempts to forecast this “one number[i.e. the year by which transformative AI will be developed]”? If one doesn’t think it allows for clearly resolvable forecasts, perhaps it would be good to note that from the outset, and when making estimates such as “more than a 10% chance of ‘transformative AI’ within 15 years”.
sidenote: There has been an argument that ‘radically transformative AI’ is a better term for the Industrial Revolution definition, given the semantic bleaching already taking place with ‘transformative AI’.