Thanks for pointing this out. I agree that I wasn’t clear about this in the post.
My hesitations have been around adopting views with timelines and risk level that are at least as concerning as the OpenPhil cluster (Holden, Ajeya, etc.) that you’re pointing at; essentially views that seem to imply that AI and things that feed into it are clearly the most important cause area.
I’m not sure Eliezer having occasionally been overconfident, but got the general shape of things right is any evidence at all against >50% AGI in 30 years or >15% chance of catastrophe this century (though it could be evidence against Eliezer’s very high risk view).
I wouldn’t go as far as no evidence at all given that my understanding is Eliezer (+ MIRI) was heavily involved in influencing the OpenPhil’s cluster’s views so it’s not entirely independent, but I agree it’s much weaker evidence for less extreme views.
Fewer ‘smart people disagree’ about the numbers in your footnote than about the more extreme view.′
I was going to say that it seems like a big difference within our community, but both clusters of views are very far away from the median pretty reasonable person and the median AI researcher. Though I suppose the latter actually isn’t far away on timelines (potentially depending on the framing?). It definitely seems to be in significant tension with how AI researchers and the general public / markets / etc. act, regardless of stated beliefs (e.g. I found it interesting how short the American public’s timelines are, compared to their actions).
Anyway, overall I think you’re right that it makes a difference but it seems like a substantive concern for both clusters of views.
The Carlsmith post you say you roughly endorse seems to have 65% on AGI in 50 years, with a 10% chance of existential catastophe overall. So I’m not sure if that means your conclusion is
[...]
The conclusion I intend to convey is something like “I’m no longer as hesitant about adopting views which are at least as concerning as >50% of AGI/TAI/APS-AI within 30 years, and >15% chance of existential catastrophe this century” which as I referred to above seem to make AI clearly the most important cause area.
Copying my current state on the object level views from another recent post:
I’m now at ~20% by 2036; my median is now ~2050 though still with a fat right tail.
My timelines shortening [due to reflecting on MATH breakthrough] should also increase my p(AI doom by 2100) a bit, though I’m still working out my views here. I’m guessing I’ll land somewhere between 20 and 60% [TBC, most of the variance is coming from working out my views and not the MATH breakthrough].
Thanks for pointing this out. I agree that I wasn’t clear about this in the post.
My hesitations have been around adopting views with timelines and risk level that are at least as concerning as the OpenPhil cluster (Holden, Ajeya, etc.) that you’re pointing at; essentially views that seem to imply that AI and things that feed into it are clearly the most important cause area.
I wouldn’t go as far as no evidence at all given that my understanding is Eliezer (+ MIRI) was heavily involved in influencing the OpenPhil’s cluster’s views so it’s not entirely independent, but I agree it’s much weaker evidence for less extreme views.
I was going to say that it seems like a big difference within our community, but both clusters of views are very far away from the median pretty reasonable person and the median AI researcher. Though I suppose the latter actually isn’t far away on timelines (potentially depending on the framing?). It definitely seems to be in significant tension with how AI researchers and the general public / markets / etc. act, regardless of stated beliefs (e.g. I found it interesting how short the American public’s timelines are, compared to their actions).
Anyway, overall I think you’re right that it makes a difference but it seems like a substantive concern for both clusters of views.
The conclusion I intend to convey is something like “I’m no longer as hesitant about adopting views which are at least as concerning as >50% of AGI/TAI/APS-AI within 30 years, and >15% chance of existential catastrophe this century” which as I referred to above seem to make AI clearly the most important cause area.
Copying my current state on the object level views from another recent post: