Thanks for this perspective! I’ve heard of the Doomsday Argument but I haven’t read the literature. My understanding was that the majority belief is that the Doomsday Argument is wrong, we just haven’t figured out why it’s wrong. I didn’t realize there was substantial literature on the problem, so I will need to do some reading!
I think it is still accurate to claim that very few sources have considered the probability of unknown risks relative to known risks. I’m mainly basing this off the Rowe & Beard literature review, which is pretty comprehensive AFAIK. Leslie and Bostrom discuss unknown risks, but without addressing their relative probabilities (at least Bostrom doesn’t, I don’t have access to Leslie’s book right now). If you know of any sources that address this that Rowe & Beard didn’t cover, I’d be happy to hear about them.
Note that it’s not just the Doomsday Argument that may give one reason for revising one’s x-risk estimates beyond what is suggested by object-level analyses of specific risks. See this post by Robin Hanson for an enumeration of the relevant types of arguments.
I am puzzled that these arguments do not seem to be influencing many of the most cited x-risk estimates (e.g. Toby’s). Is this because these arguments are thought to be clearly flawed? Or is it because people feel they just don’t know how to think about them, and so they simply ignore them? I would like to see more “reasoning transparency” about these issues.
It’s also worth noting that some of these “speculative” arguments would provide reason for revising not only overall x-risks estimates, but also estimates of specific risks. For example, as Katja Grace and Greg Lewis have noted, a misaligned intelligence explosion cannot be the Great Filter. Accordingly, insofar as the Great Filter influences one’s x-risk estimates, one should shift probability mass away from AI risk and toward risks that are plausible Great Filters.
In line with Matthew’s comment, think it’s true that several sources discuss the possibility of unknown risks or discuss the total risk level (which should presumably include unknown risks). But I’m also not aware of any sources apart from Ord and Pamlin & Armstrong which give quantitative estimates of unknown risk specifically. (If someone knows of any, please add them to my database!)
I’m also not actually sure if I know of any other sources which even provide relatively specific qualitative statements about the probability of unknown risks causing existential catastrophe. I wouldn’t count the statements from Leslie and Bostrom, for example.
So I think Michael’s claim is fair, at least if we interpret it as “no other sources appear to have clearly addressed the likelihood of unknown x-risks in particular, which implies that most others do not give unknown risks serious consideration.”
Thanks for this perspective! I’ve heard of the Doomsday Argument but I haven’t read the literature. My understanding was that the majority belief is that the Doomsday Argument is wrong, we just haven’t figured out why it’s wrong. I didn’t realize there was substantial literature on the problem, so I will need to do some reading!
I think it is still accurate to claim that very few sources have considered the probability of unknown risks relative to known risks. I’m mainly basing this off the Rowe & Beard literature review, which is pretty comprehensive AFAIK. Leslie and Bostrom discuss unknown risks, but without addressing their relative probabilities (at least Bostrom doesn’t, I don’t have access to Leslie’s book right now). If you know of any sources that address this that Rowe & Beard didn’t cover, I’d be happy to hear about them.
Note that it’s not just the Doomsday Argument that may give one reason for revising one’s x-risk estimates beyond what is suggested by object-level analyses of specific risks. See this post by Robin Hanson for an enumeration of the relevant types of arguments.
I am puzzled that these arguments do not seem to be influencing many of the most cited x-risk estimates (e.g. Toby’s). Is this because these arguments are thought to be clearly flawed? Or is it because people feel they just don’t know how to think about them, and so they simply ignore them? I would like to see more “reasoning transparency” about these issues.
It’s also worth noting that some of these “speculative” arguments would provide reason for revising not only overall x-risks estimates, but also estimates of specific risks. For example, as Katja Grace and Greg Lewis have noted, a misaligned intelligence explosion cannot be the Great Filter. Accordingly, insofar as the Great Filter influences one’s x-risk estimates, one should shift probability mass away from AI risk and toward risks that are plausible Great Filters.
In line with Matthew’s comment, think it’s true that several sources discuss the possibility of unknown risks or discuss the total risk level (which should presumably include unknown risks). But I’m also not aware of any sources apart from Ord and Pamlin & Armstrong which give quantitative estimates of unknown risk specifically. (If someone knows of any, please add them to my database!)
I’m also not actually sure if I know of any other sources which even provide relatively specific qualitative statements about the probability of unknown risks causing existential catastrophe. I wouldn’t count the statements from Leslie and Bostrom, for example.
So I think Michael’s claim is fair, at least if we interpret it as “no other sources appear to have clearly addressed the likelihood of unknown x-risks in particular, which implies that most others do not give unknown risks serious consideration.”