[I’m a contest organizer but I’m recusing myself for this because I personally know Andrew.]
Thanks for writing! A few minor points (may leave more substantive points later).
In 2014, one survey asked the 100 most cited living AI scientists by what year they saw a 10%, 50%, and 90% chance that HLMI would exist
There is updated research on this here (survey conducted 2019) and here (2022; though it’s not a paper yet, so might not be palatable for some people).
Only 17% of respondents said they were never at least 90% confident HLMI would exist.
I think this is a typo.
Considering all of these scenarios together, 80,000 Hours’ team of AI experts estimates that “the risk of a severe, even existential catastrophe caused by machine intelligence within the next 100 years is something like 10%.”
I don’t think I would cite 80,000 hours, as that particular article is older. There is a newer one recently, but it still seems better for ethos to cite something that looks like a paper. You could possibly cite Carlsmith or the survey above, which I think says the median researcher assigns 5% chance of extinction-level catastrophe.
Thanks Thomas—appreciate the updated research. And that wasn’t a typo, just a poorly expressed idea. I meant to say, “Only 17% of respondents reported less than 90% confidence that HLMI will eventually exist.”
[I’m a contest organizer but I’m recusing myself for this because I personally know Andrew.]
Thanks for writing! A few minor points (may leave more substantive points later).
There is updated research on this here (survey conducted 2019) and here (2022; though it’s not a paper yet, so might not be palatable for some people).
I think this is a typo.
I don’t think I would cite 80,000 hours, as that particular article is older. There is a newer one recently, but it still seems better for ethos to cite something that looks like a paper. You could possibly cite Carlsmith or the survey above, which I think says the median researcher assigns 5% chance of extinction-level catastrophe.
Thanks Thomas—appreciate the updated research. And that wasn’t a typo, just a poorly expressed idea. I meant to say, “Only 17% of respondents reported less than 90% confidence that HLMI will eventually exist.”