“(The first public ‘general AI’ system is predicted in 2038, which makes me a bit confused. I fail to see how there’s an 11 year gap between weak and ‘strong’ AI, especially with superintelligence ~10 months after the first AGI. Am I missing something?).”
Nice that you noticed this! This is, I think, an inconsistency in Metaculus’ answers, one that has been pointed out at least twice before but still hasn’t corrected itself.
No one has ever demonstrated that crowdsourced forecasts have any kind of validity over these kinds of timelines, especially when applied to something to no base rate!!! I have no idea why anyone takes the Metaculus outputs at all seriously when it comes to this kind of thing. I don’t want to say it’s the same as consulting the local fortune-telling gypsy but it might not be far off!
It’s all relative. I trust my own forecasts more than Metaculus’ forecasts (in some domains; in other domains I don’t) because I’ve thought through the arguments for myself. But for topics I know little about, who would I trust more—Metaculus or my Twitter feed or some op-ed in the New York Times? Answer: Metaculus.
Obviously I’d drop Metaculus in an instant if I had a better source of evidence, it’s just that often I don’t.
No, it is not all relative imo. If you know nothing, you just know nothing. If all you have is noise, you don’t have any signal. If Metaculus and the NYT are both generating pure noise they’re just worth the same amount: zero. Metaculus’s noise might well feel a lot better because the outputs are generated by honest humans doing their best, whereas the NYT is constantly trying to lie to you all the time, but some things are just not forecastable, I’m sorry. Neither Metaculus nor the NYT or any pool of economists is ever going to generate any kind of good Brier score when it comes to one months’ worth of nonfarm payrolls data because it’s just inherently incredibly noisy. This is a fairly extreme example, but the principle applies much more broadly.
I mean, yeah, some things are basically impossible to get any signal/evidence/etc. on and for those things the NYT and Metaculus and the best forecaster in the world are all equally useless.
But predicting AGI isn’t one of those topics. It instead is one of the vast majority of topics where rational analysis, rational gathering of evidence, etc. can pay dividends.
Is it? Imo predicting development of novel technologies in general is unbelievably hard (there’s no base rate!) and AGI in particular seems even harder to do. Go back 5 years and I’m quite confident that everyone would have done absolutely horribly at predicting the course of current AI developments, both over-optimistic (whoooo remembers the self-driving car hype) and over-pessimistic (v much doubt anyone would have thought that image generation would be so good!). And AGI seems even harder to forecast
I didn’t say it was easy! I just said that rational analysis, rational gathering of evidence, etc. can pay dividends.
And indeed, if you go back 5 years and look at what people were saying at the time, some people did do way better than most at predicting what happened.* I happen to remember being at dinner parties in the Bay in late 2018, early 2019 where LWers were discussing the topic of “If, as now seems quite plausible, predicting text is the key to general intelligence & will scale to AGI, what implications does that have?” This may even have been before GPT-2 was public, I don’t remember. Probably it was shortly after.
That’s on hard mode though—to prove my point all I have to do is point out that most of the world has been surprised by the general pace of progress in AI, and in particular progress towards AGI, in the last 5 years. It wasn’t even on the radar for most people. But for some people not only was it on the radar but it was basically what they expected. (MIRI’s timelines haven’t changed much in the last 5 years, I hear, because things have more or less proceeded about as quickly as they thought. Different in the details of course, but not generally slower or faster.)
*And I don’t think they just got lucky. They were well-connected and following the field closely, and took the forecasting job unusually seriously, and were unusually rational as people.
“(The first public ‘general AI’ system is predicted in 2038, which makes me a bit confused. I fail to see how there’s an 11 year gap between weak and ‘strong’ AI, especially with superintelligence ~10 months after the first AGI. Am I missing something?).”
Nice that you noticed this! This is, I think, an inconsistency in Metaculus’ answers, one that has been pointed out at least twice before but still hasn’t corrected itself.
No one has ever demonstrated that crowdsourced forecasts have any kind of validity over these kinds of timelines, especially when applied to something to no base rate!!! I have no idea why anyone takes the Metaculus outputs at all seriously when it comes to this kind of thing. I don’t want to say it’s the same as consulting the local fortune-telling gypsy but it might not be far off!
It’s all relative. I trust my own forecasts more than Metaculus’ forecasts (in some domains; in other domains I don’t) because I’ve thought through the arguments for myself. But for topics I know little about, who would I trust more—Metaculus or my Twitter feed or some op-ed in the New York Times? Answer: Metaculus.
Obviously I’d drop Metaculus in an instant if I had a better source of evidence, it’s just that often I don’t.
No, it is not all relative imo. If you know nothing, you just know nothing. If all you have is noise, you don’t have any signal. If Metaculus and the NYT are both generating pure noise they’re just worth the same amount: zero. Metaculus’s noise might well feel a lot better because the outputs are generated by honest humans doing their best, whereas the NYT is constantly trying to lie to you all the time, but some things are just not forecastable, I’m sorry. Neither Metaculus nor the NYT or any pool of economists is ever going to generate any kind of good Brier score when it comes to one months’ worth of nonfarm payrolls data because it’s just inherently incredibly noisy. This is a fairly extreme example, but the principle applies much more broadly.
I mean, yeah, some things are basically impossible to get any signal/evidence/etc. on and for those things the NYT and Metaculus and the best forecaster in the world are all equally useless.
But predicting AGI isn’t one of those topics. It instead is one of the vast majority of topics where rational analysis, rational gathering of evidence, etc. can pay dividends.
Is it? Imo predicting development of novel technologies in general is unbelievably hard (there’s no base rate!) and AGI in particular seems even harder to do. Go back 5 years and I’m quite confident that everyone would have done absolutely horribly at predicting the course of current AI developments, both over-optimistic (whoooo remembers the self-driving car hype) and over-pessimistic (v much doubt anyone would have thought that image generation would be so good!). And AGI seems even harder to forecast
I didn’t say it was easy! I just said that rational analysis, rational gathering of evidence, etc. can pay dividends.
And indeed, if you go back 5 years and look at what people were saying at the time, some people did do way better than most at predicting what happened.* I happen to remember being at dinner parties in the Bay in late 2018, early 2019 where LWers were discussing the topic of “If, as now seems quite plausible, predicting text is the key to general intelligence & will scale to AGI, what implications does that have?” This may even have been before GPT-2 was public, I don’t remember. Probably it was shortly after.
That’s on hard mode though—to prove my point all I have to do is point out that most of the world has been surprised by the general pace of progress in AI, and in particular progress towards AGI, in the last 5 years. It wasn’t even on the radar for most people. But for some people not only was it on the radar but it was basically what they expected. (MIRI’s timelines haven’t changed much in the last 5 years, I hear, because things have more or less proceeded about as quickly as they thought. Different in the details of course, but not generally slower or faster.)
*And I don’t think they just got lucky. They were well-connected and following the field closely, and took the forecasting job unusually seriously, and were unusually rational as people.