I disagree, and think that you can’t take a claim that there’s more than 10% chance in 14 years as evidence of your claim that there’s a consensus of 10% or more in 10 years or less.
Also the fact that that Holden makes that estimate doesn’t show that there’s a broad consensus in support of that estimate. “Consensus” means “a generally accepted opinion; wide agreement”. And the fact that Holden has come to a particular conclusion based on having talked to many experts doesn’t show that there is wide agreement among those experts.
These kinds of estimates are imprecise by nature, so perhaps I should change “≥10% probability of AGI in ≤10 years” to “~10% within 10 years? (To me at least, more than 10% in 14 years translates to something like 6 or 7% in 10 years, which I don’t think would alter my actions much. I don’t think crossing the “less than a decade to sort this out” line is dependent on the estimate a decade out being firmly in the double-figure percents by all accounts (rather, I think “likely-at-least-close-to-10%” by an increasing number of knowledgeable value-aligned people, is more than enough for action)).
Holden’s estimate is based on the series of in depth reports (linked above), reviewed by multiple experts, each of which comes to similar conclusions. I’ll note that I said “increasingly broad consensus”, which is a somewhat ambiguous phrasing. Would it help if I changed it to “consensus forming”?
We will be posting a follow up of Grace et al from a 2019 survey soon. Can link here once it is up. I’ll also make sure the figures for shorter years from now or lower percent probability are noted in the article somewhere.
Edit: For what it’s worth it is around 8% by 2032 in this sample.
Preprint is up and can be found here. Table S7 in the appendix may be particularly useful to answer some of the above. There will be two new surveys this year that gather new data on HLMI forecasts and the results will be out a lot faster this time round.
I disagree, and think that you can’t take a claim that there’s more than 10% chance in 14 years as evidence of your claim that there’s a consensus of 10% or more in 10 years or less.
Also the fact that that Holden makes that estimate doesn’t show that there’s a broad consensus in support of that estimate. “Consensus” means “a generally accepted opinion; wide agreement”. And the fact that Holden has come to a particular conclusion based on having talked to many experts doesn’t show that there is wide agreement among those experts.
These kinds of estimates are imprecise by nature, so perhaps I should change “≥10% probability of AGI in ≤10 years” to “~10% within 10 years? (To me at least, more than 10% in 14 years translates to something like 6 or 7% in 10 years, which I don’t think would alter my actions much. I don’t think crossing the “less than a decade to sort this out” line is dependent on the estimate a decade out being firmly in the double-figure percents by all accounts (rather, I think “likely-at-least-close-to-10%” by an increasing number of knowledgeable value-aligned people, is more than enough for action)).
Holden’s estimate is based on the series of in depth reports (linked above), reviewed by multiple experts, each of which comes to similar conclusions. I’ll note that I said “increasingly broad consensus”, which is a somewhat ambiguous phrasing. Would it help if I changed it to “consensus forming”?
I think you should gather more data (e.g. via surveys) on what credences experts would assign to AGI within 10 years.
I’m not sure if I’m the best person to do this. Would be good to see AI Impacts do a follow-up to their survey of 6 years ago.
We will be posting a follow up of Grace et al from a 2019 survey soon. Can link here once it is up. I’ll also make sure the figures for shorter years from now or lower percent probability are noted in the article somewhere.
Edit: For what it’s worth it is around 8% by 2032 in this sample.
Cool, thanks!
Preprint is up and can be found here. Table S7 in the appendix may be particularly useful to answer some of the above. There will be two new surveys this year that gather new data on HLMI forecasts and the results will be out a lot faster this time round.
Thanks! Here is Table S7 (I’ve highlighted the relevant years):
I’m thinking that it would be good to have a survey with the following 2 questions:
1. Year with 10% chance of AGI.
2. P(doom|AGI in that year).