if the argument of AI2027 is âwe shouldnât dismiss this outcome out of handâ then itâs a reasonable position
Yep, that is how Titotal summarizes the argument:
The scenario in the short story is not the median forecast for any AI futures author, and none of the AI2027 authors actually believe that 2027 is the median year for a singularity to happen. But the argument they make is that 2027 is a plausible year
And if titotal had ended their post with something like â⌠and so I think 2027 is a bit less plausible than the authors doâ I would have no confusion. But they ended with:
What Iâm most against is people taking shoddy toy models seriously and basing life decisions on them, as I have seen happen for AI2027
And I therefore am left wondering what less shoddy toy models I should be basing my life decisions on.[1]
I think their answer is partly ânaively extrapolating the METR time horizon numbers forward is better than AI 2027â? But I donât want to put words in their mouth and also I interpret them to have much longer timelines than this naive extrapolation would imply.
I think less selective quotation makes the line of argument clear.
Continuing the first quote:
The scenario in the short story is not the median forecast for any AI futures author, and none of the AI2027 authors actually believe that 2027 is the median year for a singularity to happen. But the argument they make is that 2027 is a plausible year,and they back it up with images of sophisticated looking modelling like the following:
[img]
This combination of compelling short story and seemingly-rigorous research may have been the secret sauce that let the article to go viral and be treated as a serious project:
[quote]
Now, I was originally happy to dismiss this work and just wait for their predictions to fail, but this thing just keeps spreading, including a youtube video with millions of views. So I decided to actually dig into the model and the code, and try to understand what the authors were saying and what evidence they were using to back it up.
The article is huge, so I focussed on one section alone: their âtimelines forecastâ code and accompanying methodology section. Not to mince words, I think itâs pretty bad. Itâs not just that I disagree with their parameter estimates, itâs that I think the fundamental structure of their model is highly questionable and at times barely justified, there is very little empirical validation of the model, and there are parts of the code that the write-up of the model straight up misrepresents.
So the summary of this would not be â⌠and so I think AI 2027 is a bit less plausible than the authors doâ, but something like: âI think the work motivating AI 2027 being a credible scenario is, in fact, not good, and should not persuade those who did not believe this already. It is regrettable this work is being publicised (and perhaps presented) as much stronger than it really is.â
Continuing the second quote:
What Iâm most against is people taking shoddy toy models seriously and basing life decisions on them, as I have seen happen for AI2027. This is just a model for a tiny slice of the possibility space for how AI will go, and in my opinion it is implemented poorly even if you agree with the authorâs general worldview.
The right account for decision making under (severe) uncertainty is up for grabs, but in the âmake a less shoddy toy modelâ approach the quote would urge having a wide ensemble of different ones (including, say, those which are sub-exponential, âhit the wallâ or whatever else), and further urge we should put very little weight on the AI2027 model in whatever ensemble we will be using for important decisions.
Titotal actually ended their post with an alternative prescription:
I think people are going to deal with the fact that itâs really difficult to predict how a technology like AI is going to turn out. The massive blobs of uncertainty shown in AI 2027 are still severe underestimates of the uncertainty involved. If your plans for the future rely on prognostication, and this is the standard of work you are using, I think your plans are doomed. I would advise looking into plans that are robust to extreme uncertainty in how AI actually goes, and avoid actions that could blow up in your face if you turn out to be badly wrong.
I would advise looking into plans that are robust to extreme uncertainty in how AI actually goes, and avoid actions that could blow up in your face if you turn out to be badly wrong.
Seeing you highlight this now it occurs to me that I basically agree with this w.r.t. AI timelines (at least on one plausible interpretation, my guess is that titotal could have a different meaning in mind). I mostly donât think people should take actions that blow up in their face if timelines are long (there are some exceptions, but overall I think long timelines are plausible and actions should be taken with that in mind).
A key thing that titotal doesnât mention is how much probability mass they put on short timelines like, say, AGI by 2030. This seems very important for weighing various actions, even though we both agree that we should also be prepared for longer timelines.
In general, I feel like executing plans that are robust to extreme uncertainty is a prescription that is hard to follow without having at least a vague idea of the distribution of likelihood of various possibilities.
Thanks! This is helpful, although I would still be interested to hear if they believe there are models with âhave strong conceptual justifications or empirical validation with existing dataâ.
I was going to reply with something longer here, but I think Gregory Lewisâ excellent comment highlights most of what I wanted to, r.e. titotal does actually give an alternative suggestion in the piece.
So instead Iâll counter two claims I think you make (or imply) in your comments here:
1. A shoddy toy model is better than no model at all
I mean this seems clearly not true, if we take model to be referring to the sort of formalised, quantified exercise similar to AI-2027. Some examples here might be Samuelsonâs infamous predictions of the Soviet Union inevitably overtaking the US in GNP.[1] This was a bad model of the world, and even if it was âbetterâ than the available alternatives or came from a more prestigious source, it was still bad and I think worse than no model (again, defined as formal exercise ala AI2027).
A second example I can think of is the infamous Growth in a Time of Debtpaper, which I remember being used to win arguments and justify austerity across Europe in the 2010s, being rendered much less convincing after an Excel error was corrected.[2]
This also seems clearly false, unless weâre stretching âmodelâ to mean simply âa reason/âargument/âjustificationâ or defining âlife decisionsâ narrowly as only those with enormous consequences instead of any âdecision about my lifeâ.
Even in the more serious cases, the role of models is to support presenting arguments for or against some decision or not, or to frame some explanation about the world, and of course simplification and quantification can be useful and powerful, but they shouldnât be the only game in town. Other schools of thought are available.[3]
The reproduction paper turned critique is here, feels crazy that I canât see the original data but the âmodelâ here seemed just to be spreadsheet of ~20 countries where the average only counted 15
This also seems clearly false, unless weâre stretching âmodelâ to mean simply âa reason/âargument/âjustificationâ
Yep, this is what I meant, sorry for the confusion. Or to phrase it another way: âIâm going off my intuitionâ is not a type of model which has privileged epistemic status; itâs one which can be compared with something like AI 2027 (and, like you say, may be found better).
Besides the point that âshoddy toy modelsâ might be emotionally charged, I just want to point out that accelerating progress majorly increases variance and unknown unknowns? The higher energy a system is and the more variables you have the more chaotic it becomes. So maybe an answer is that a agile short-range model is the best? Outside view it in moderation and plan with the next few years being quite difficult to predict?
You donât really need another model to disprove an existing one, you might as well point out that we donât know and that is okay too.
Yep, that is how Titotal summarizes the argument:
And if titotal had ended their post with something like â⌠and so I think 2027 is a bit less plausible than the authors doâ I would have no confusion. But they ended with:
And I therefore am left wondering what less shoddy toy models I should be basing my life decisions on.[1]
I think their answer is partly ânaively extrapolating the METR time horizon numbers forward is better than AI 2027â? But I donât want to put words in their mouth and also I interpret them to have much longer timelines than this naive extrapolation would imply.
I think less selective quotation makes the line of argument clear.
Continuing the first quote:
So the summary of this would not be â⌠and so I think AI 2027 is a bit less plausible than the authors doâ, but something like: âI think the work motivating AI 2027 being a credible scenario is, in fact, not good, and should not persuade those who did not believe this already. It is regrettable this work is being publicised (and perhaps presented) as much stronger than it really is.â
Continuing the second quote:
The right account for decision making under (severe) uncertainty is up for grabs, but in the âmake a less shoddy toy modelâ approach the quote would urge having a wide ensemble of different ones (including, say, those which are sub-exponential, âhit the wallâ or whatever else), and further urge we should put very little weight on the AI2027 model in whatever ensemble we will be using for important decisions.
Titotal actually ended their post with an alternative prescription:
Seeing you highlight this now it occurs to me that I basically agree with this w.r.t. AI timelines (at least on one plausible interpretation, my guess is that titotal could have a different meaning in mind). I mostly donât think people should take actions that blow up in their face if timelines are long (there are some exceptions, but overall I think long timelines are plausible and actions should be taken with that in mind).
A key thing that titotal doesnât mention is how much probability mass they put on short timelines like, say, AGI by 2030. This seems very important for weighing various actions, even though we both agree that we should also be prepared for longer timelines.
In general, I feel like executing plans that are robust to extreme uncertainty is a prescription that is hard to follow without having at least a vague idea of the distribution of likelihood of various possibilities.
Thanks! This is helpful, although I would still be interested to hear if they believe there are models with âhave strong conceptual justifications or empirical validation with existing dataâ.
I was going to reply with something longer here, but I think Gregory Lewisâ excellent comment highlights most of what I wanted to, r.e. titotal does actually give an alternative suggestion in the piece.
So instead Iâll counter two claims I think you make (or imply) in your comments here:
1. A shoddy toy model is better than no model at all
I mean this seems clearly not true, if we take model to be referring to the sort of formalised, quantified exercise similar to AI-2027. Some examples here might be Samuelsonâs infamous predictions of the Soviet Union inevitably overtaking the US in GNP.[1] This was a bad model of the world, and even if it was âbetterâ than the available alternatives or came from a more prestigious source, it was still bad and I think worse than no model (again, defined as formal exercise ala AI2027).
A second example I can think of is the infamous Growth in a Time of Debt paper, which I remember being used to win arguments and justify austerity across Europe in the 2010s, being rendered much less convincing after an Excel error was corrected.[2]
TL;dr, as Thane said on LessWrong, we shouldnât grade models on a curve
2. You need to base life decisions on a toy model
This also seems clearly false, unless weâre stretching âmodelâ to mean simply âa reason/âargument/âjustificationâ or defining âlife decisionsâ narrowly as only those with enormous consequences instead of any âdecision about my lifeâ.
Even in the more serious cases, the role of models is to support presenting arguments for or against some decision or not, or to frame some explanation about the world, and of course simplification and quantification can be useful and powerful, but they shouldnât be the only game in town. Other schools of thought are available.[3]
The reproduction paper turned critique is here, feels crazy that I canât see the original data but the âmodelâ here seemed just to be spreadsheet of ~20 countries where the average only counted 15
Such as:
Decisionmaking under Deep Uncertainty
Do The Math, Then Burn The Math and Go With Your Gut
Make a decision based on the best explanation of the world
Go with common-sense heuristics since they likely encode knowledge gained from cultural evolution
Yep, this is what I meant, sorry for the confusion. Or to phrase it another way: âIâm going off my intuitionâ is not a type of model which has privileged epistemic status; itâs one which can be compared with something like AI 2027 (and, like you say, may be found better).
Besides the point that âshoddy toy modelsâ might be emotionally charged, I just want to point out that accelerating progress majorly increases variance and unknown unknowns? The higher energy a system is and the more variables you have the more chaotic it becomes. So maybe an answer is that a agile short-range model is the best? Outside view it in moderation and plan with the next few years being quite difficult to predict?
You donât really need another model to disprove an existing one, you might as well point out that we donât know and that is okay too.