I’m curious whether there’s any answer AI experts could have given that would be a reasonably big update for you.
For example is there any level of consensus against ~AGI by 2070 (or some other date) that would be strong enough to move your forecast by 10 percentage points?
Good question. I think AI researchers views inform/can inform me. A few examples from the recent NLP Community Metasurvey. I would quote bits from this summary.
Few scaling maximalists: 17% agreed that Given resources (i.e., compute and data) that could come to exist this century, scaled-up implementations of established existing techniques will be sufficient to practically solve any important real-world problem or application in NLP.
This was surprsing and updated me somewhat against shorter timelines (and higher risk) as, for example, it clashes with the “+12 OOMS Enough” premise of the Kokotajlo’s argument for short timelines of the Carlsmith report (see also this and his review of Carlsmith report).
NLP is on a path to AGI: 58% agreed that Understanding the potential development of artificial general intelligence (AGI) and the benefits/risks associated with it should be a significant priority for NLP researchers.
Related: 57% agreed that Recent developments in large-scale ML modeling (such as in language modeling and reinforcement learning) are significant steps toward the development of AGI.
If these numbers were significantly lower or higher, it would also probably update my views.
AGI could be catastrophic: 36% agreed that It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war.
46% of women and 53% of URM respondents agreed.
The comments suggested that people took a pretty wide range of interpretations to this, including things like OOD robustness failures leading to weapons launches.
This number is puzzling and hard to interpret. It seems appropriate in light of AI Impacts’ What do ML researchers think about AI in 2022? where “48% of respondents gave at least 10% chance of an extremely bad outcome”.
I don’t fully understand what this implies about the ML community’s views on AI alignment. But I can see myself updating positively if these concerns would lead to more safety culture, alignment research, etc.
Note that we’d probably also look at the object level reasons for why they think that. E.g., new scaling laws findings could definitely shift our/my forecast by 10%.
Fair question. I say little weight but if it was far enough from my view I would update a little. My view also may not be representative of other forecasters, as is evident from Misha’s comment.
For example is there any level of consensus against ~AGI by 2070 (or some other date) that would be strong enough to move your forecast by 10 percentage points?
From the original Grace et al. survey (and I think the more recent ones as well? but haven’t read as closely) the ML researchers clearly had very incoherent views depending on the question being asked and elicitation techniques, which I think provides some evidence they haven’t thought about it that deeply and we shouldn’t take it too seriously (some incoherence is expected, but I think they gave wildly different answers for HLMI (human-level machine intelligence) and full automation of labor).
So I think I’d split up the thresholds by somewhat coherent vs. still very incoherent.
My current forecast for ~AGI by 2100 barring pre-AGI catastrophe is 80%. To move it to 70% based just a survey of ML experts, I think I’d have to see something like one of:
ML experts still appear to be very incoherent, but are giving a ~10% chance of ~AGI by 2100 on average across framings.
ML experts appear to be somewhat coherent, and are giving a ~25% chance of ~AGI by 2100.
(but I haven’t thought about this a lot, these numbers could change substantially on reflection or discussion/debate)
I’m curious whether there’s any answer AI experts could have given that would be a reasonably big update for you.
For example is there any level of consensus against ~AGI by 2070 (or some other date) that would be strong enough to move your forecast by 10 percentage points?
Good question. I think AI researchers views inform/can inform me. A few examples from the recent NLP Community Metasurvey. I would quote bits from this summary.
This was surprsing and updated me somewhat against shorter timelines (and higher risk) as, for example, it clashes with the “+12 OOMS Enough” premise of the Kokotajlo’s argument for short timelines of the Carlsmith report (see also this and his review of Carlsmith report).
If these numbers were significantly lower or higher, it would also probably update my views.
This number is puzzling and hard to interpret. It seems appropriate in light of AI Impacts’ What do ML researchers think about AI in 2022? where “48% of respondents gave at least 10% chance of an extremely bad outcome”.
I don’t fully understand what this implies about the ML community’s views on AI alignment. But I can see myself updating positively if these concerns would lead to more safety culture, alignment research, etc.
Note that we’d probably also look at the object level reasons for why they think that. E.g., new scaling laws findings could definitely shift our/my forecast by 10%.
Fair question. I say little weight but if it was far enough from my view I would update a little. My view also may not be representative of other forecasters, as is evident from Misha’s comment.
From the original Grace et al. survey (and I think the more recent ones as well? but haven’t read as closely) the ML researchers clearly had very incoherent views depending on the question being asked and elicitation techniques, which I think provides some evidence they haven’t thought about it that deeply and we shouldn’t take it too seriously (some incoherence is expected, but I think they gave wildly different answers for HLMI (human-level machine intelligence) and full automation of labor).
So I think I’d split up the thresholds by somewhat coherent vs. still very incoherent.
My current forecast for ~AGI by 2100 barring pre-AGI catastrophe is 80%. To move it to 70% based just a survey of ML experts, I think I’d have to see something like one of:
ML experts still appear to be very incoherent, but are giving a ~10% chance of ~AGI by 2100 on average across framings.
ML experts appear to be somewhat coherent, and are giving a ~25% chance of ~AGI by 2100.
(but I haven’t thought about this a lot, these numbers could change substantially on reflection or discussion/debate)