There’s a grain that I agree with here, which is that people excessively plan around a median year for AGI rather than a distribution for various events, and that planning around that kind of distribution leads to more robust and high-expected-value actions (and perhaps less angst). However, I strongly disagree with the idea that we already know “what we need.” Off the top of my head, several ways narrowing the error bars on timelines—which I’ll operationalize as “the distribution of the most important decisions with respect to building transformative AI”—would be incredibly useful:
To what extent will these decisions be made by the current US administration, or by people governed by the current administration? This affects the political strategy everyone—including, I propose, PauseAI—should adopt.
To what extent will the people making the most important AI decisions remember stuff people said in 2025? This is very important for the relative usefulness of public communications versus research, capacity-building, etc.
Are these decisions soon enough that the costs of being “out of the action” outweigh the longer-term benefits of e.g. going to grad school, developing technical expertise, etc? Clearly relevant for lots of individuals who want to make a big impact.
When should philanthropists spend their resources? As I and others have written, there are several considerations that point towards spending later; these are weakened a lot if the key decisions are in the next few years.
To what extent will the most transformative models be technically similar to the ones we have today? That answer determines the value of technical safety research.
I also strongly disagree with the framing that the important thing is us knowing what we know. Yes, people who have been immersed in AI content for years often believe that very scary and/or awesome AI capabilities are coming within the decade. But most people, including most of the people who might take the most important actions, are not in this category and do not share this view (or at least don’t seem to have internalized it). Work that provides an empirical grounding for AI forecasts has already been very useful in bringing attention to AGI and its risks from a broader set of people, including in governments, who would otherwise be focused on any one of the million other problems in the world.
I agree that not everyone already knows what they need to know. Our crux issue is probably “who needs to get it and how will they learn it?” I think we more than have the evidence to teach and set an example of knowing for the public. I think you think we need to make a very respectable and detailed case to convince elites. I think you can take multiple routes to influencing elites and that they will be more receptive when the reality of AI risk is a more popular view. I don’t think timelines are a great tool for convincing either of these groups because they create such a sense of panic and there’s such an invitation to quibble with the forecasts instead of facing the thrust of the evidence.
I definitely agree there are plenty of ways we should reach elites and non-elites alike that aren’t statistical models of timelines, and insofar as the resources going towards timeline models (in terms of talent, funding, bandwidth) are fungible with the resources going towards other things, maybe I agree that more effort should be going towards the other things (but I’m not sure—I really think the timeline models have been useful for our community’s strategy and for informing other audiences).
But also, they only sometimes create a sense of panic; I could see specificity being helpful for people getting out of the mode of “it’s vaguely inevitable, nothing to be done, just gotta hope it all works out.” (Notably the timeline models sometimes imply longer timelines than the vibes coming out of the AI companies and Bay Area house parties.)
There’s a grain that I agree with here, which is that people excessively plan around a median year for AGI rather than a distribution for various events, and that planning around that kind of distribution leads to more robust and high-expected-value actions (and perhaps less angst).
However, I strongly disagree with the idea that we already know “what we need.” Off the top of my head, several ways narrowing the error bars on timelines—which I’ll operationalize as “the distribution of the most important decisions with respect to building transformative AI”—would be incredibly useful:
To what extent will these decisions be made by the current US administration, or by people governed by the current administration? This affects the political strategy everyone—including, I propose, PauseAI—should adopt.
To what extent will the people making the most important AI decisions remember stuff people said in 2025? This is very important for the relative usefulness of public communications versus research, capacity-building, etc.
Are these decisions soon enough that the costs of being “out of the action” outweigh the longer-term benefits of e.g. going to grad school, developing technical expertise, etc? Clearly relevant for lots of individuals who want to make a big impact.
When should philanthropists spend their resources? As I and others have written, there are several considerations that point towards spending later; these are weakened a lot if the key decisions are in the next few years.
To what extent will the most transformative models be technically similar to the ones we have today? That answer determines the value of technical safety research.
I also strongly disagree with the framing that the important thing is us knowing what we know. Yes, people who have been immersed in AI content for years often believe that very scary and/or awesome AI capabilities are coming within the decade. But most people, including most of the people who might take the most important actions, are not in this category and do not share this view (or at least don’t seem to have internalized it). Work that provides an empirical grounding for AI forecasts has already been very useful in bringing attention to AGI and its risks from a broader set of people, including in governments, who would otherwise be focused on any one of the million other problems in the world.
I agree that not everyone already knows what they need to know. Our crux issue is probably “who needs to get it and how will they learn it?” I think we more than have the evidence to teach and set an example of knowing for the public. I think you think we need to make a very respectable and detailed case to convince elites. I think you can take multiple routes to influencing elites and that they will be more receptive when the reality of AI risk is a more popular view. I don’t think timelines are a great tool for convincing either of these groups because they create such a sense of panic and there’s such an invitation to quibble with the forecasts instead of facing the thrust of the evidence.
I definitely agree there are plenty of ways we should reach elites and non-elites alike that aren’t statistical models of timelines, and insofar as the resources going towards timeline models (in terms of talent, funding, bandwidth) are fungible with the resources going towards other things, maybe I agree that more effort should be going towards the other things (but I’m not sure—I really think the timeline models have been useful for our community’s strategy and for informing other audiences).
But also, they only sometimes create a sense of panic; I could see specificity being helpful for people getting out of the mode of “it’s vaguely inevitable, nothing to be done, just gotta hope it all works out.” (Notably the timeline models sometimes imply longer timelines than the vibes coming out of the AI companies and Bay Area house parties.)