I agree with a lot of this post. In particular, getting more precision in timelines is probably not going to help much with persuading most people, or in influencing most of the high-level strategic questions that Miles mentions. I also expect that it’s going to be hard to get much better predictions than we have now: much of the low-hanging fruit has been plucked. However, I’d personally find better timelines quite useful for prioritizing my technical research agenda problems to work on. I might be in a minority here, but I suspect not that small a one (say 25-50% of AI safety researchers).
There’s two main ways timelines influence what I would want to work on. First, it directly changes the “deadline” I am working towards. If I thought the deadline was 5 years, I’d probably work on scaling up the most promising approaches we have now—warts and all. If I thought it was 10 years away, I’d try and make conceptual progress that could be scaled in the future. If it was 20 years away, I’d focus more on longer-term field building interventions: clarifying what the problems are, helping develop good community epistemics, mentoring people, etc. I do think what matters here is something like the log-deadline more than the deadline itself (5 vs 10 is very decision relevant, 20 vs 25 much less so) which we admittedly have a better sense of, although there’s still some considerable disagreement.
The second way timelines are relevant is that my prediction on how AI is developed changes a lot conditioned on timelines. I think we should probably just try to forecast or analyze how-AI-is-developed directly—but timelines are perhaps easier to formalize. If timelines are less than 10 years I’d be confident we develop it within the current deep learning paradigm. More than that and possibilities open up a lot. So overall longer timelines would push me towards more theoretical work (that’s generally applicable across a range of paradigms) and taking bets on underdog areas of ML . There’s not much research into, say, how to align an AI built on top of a probabilistic programming language. I’d say that’s probably not a good use of resources right now—but if we had a confident prediction human-level AI was 50 years away, I might change my mind.
You could argue from a “flash of insight” and scientific paradigm shifts generally giving rise to sudden progress. We certainly know contemporary techniques are vastly less sample and compute efficient than the human brain—so there does exist some learning algorithm much better than what we have today. Moreover there probably exists some learning algorithm that would give rise to AGI on contemporary (albeit expensive) hardware. For example, ACX notes there’s a supercomputer than can do $10^17$ FLOPS vs the estimated $10^16 needed for a human brain. These kinds of comparisons are always a bit apples to oranges, but it does seem like compute is probably not the bottleneck (or won’t be in 10 years) for a maximally-efficient algorithm.
The nub of course is whether such an algorithm is plausibly reachable by human flash of insight (and not via e.g. detailed empirical study and refinement of a less efficient but working AGI). It’s hard to rule out. How simple/universal we think the algorithm the human brain implements is one piece of evidence here—the more complex and laden with inductive bias (e.g. innate behavior), the less likely we are to come up with it. But even if the human brain is a Rube Goldberg machine, perhaps there does exist some more straightforward algorithm evolution did not happen upon.
Personally I’d put little weight on this. I have <10% probability on AGI in next 10 years, and think I put no more than 15% on AGI being developed ever by something that looks like a sudden insight than more continuous progress. Notably even if such an insight does happen soon, I’d expect it to take at least 3-5 years for it to gain recognition and be sufficiently scaled up to work. I do think it’s probable enough for us to actively keep an eye out for promising new ideas that could lead to AGI so we can be ahead of the game. I think it’s good for example that a lot of people working on AI safety were working on language models “before it was cool” (I was not one of these people), for example, although we’ve maybe now piled too much into that area.
I agree with a lot of this post. In particular, getting more precision in timelines is probably not going to help much with persuading most people, or in influencing most of the high-level strategic questions that Miles mentions. I also expect that it’s going to be hard to get much better predictions than we have now: much of the low-hanging fruit has been plucked. However, I’d personally find better timelines quite useful for prioritizing my technical research agenda problems to work on. I might be in a minority here, but I suspect not that small a one (say 25-50% of AI safety researchers).
There’s two main ways timelines influence what I would want to work on. First, it directly changes the “deadline” I am working towards. If I thought the deadline was 5 years, I’d probably work on scaling up the most promising approaches we have now—warts and all. If I thought it was 10 years away, I’d try and make conceptual progress that could be scaled in the future. If it was 20 years away, I’d focus more on longer-term field building interventions: clarifying what the problems are, helping develop good community epistemics, mentoring people, etc. I do think what matters here is something like the log-deadline more than the deadline itself (5 vs 10 is very decision relevant, 20 vs 25 much less so) which we admittedly have a better sense of, although there’s still some considerable disagreement.
The second way timelines are relevant is that my prediction on how AI is developed changes a lot conditioned on timelines. I think we should probably just try to forecast or analyze how-AI-is-developed directly—but timelines are perhaps easier to formalize. If timelines are less than 10 years I’d be confident we develop it within the current deep learning paradigm. More than that and possibilities open up a lot. So overall longer timelines would push me towards more theoretical work (that’s generally applicable across a range of paradigms) and taking bets on underdog areas of ML . There’s not much research into, say, how to align an AI built on top of a probabilistic programming language. I’d say that’s probably not a good use of resources right now—but if we had a confident prediction human-level AI was 50 years away, I might change my mind.
Is there an argument for a <10 years timeline that doesn’t go directly through the claim that it’s going to be achieved in the current paradigm?
You could argue from a “flash of insight” and scientific paradigm shifts generally giving rise to sudden progress. We certainly know contemporary techniques are vastly less sample and compute efficient than the human brain—so there does exist some learning algorithm much better than what we have today. Moreover there probably exists some learning algorithm that would give rise to AGI on contemporary (albeit expensive) hardware. For example, ACX notes there’s a supercomputer than can do $10^17$ FLOPS vs the estimated $10^16 needed for a human brain. These kinds of comparisons are always a bit apples to oranges, but it does seem like compute is probably not the bottleneck (or won’t be in 10 years) for a maximally-efficient algorithm.
The nub of course is whether such an algorithm is plausibly reachable by human flash of insight (and not via e.g. detailed empirical study and refinement of a less efficient but working AGI). It’s hard to rule out. How simple/universal we think the algorithm the human brain implements is one piece of evidence here—the more complex and laden with inductive bias (e.g. innate behavior), the less likely we are to come up with it. But even if the human brain is a Rube Goldberg machine, perhaps there does exist some more straightforward algorithm evolution did not happen upon.
Personally I’d put little weight on this. I have <10% probability on AGI in next 10 years, and think I put no more than 15% on AGI being developed ever by something that looks like a sudden insight than more continuous progress. Notably even if such an insight does happen soon, I’d expect it to take at least 3-5 years for it to gain recognition and be sufficiently scaled up to work. I do think it’s probable enough for us to actively keep an eye out for promising new ideas that could lead to AGI so we can be ahead of the game. I think it’s good for example that a lot of people working on AI safety were working on language models “before it was cool” (I was not one of these people), for example, although we’ve maybe now piled too much into that area.