This seems very misleading. We know that COVID-19 has <<5% IFR. Presumably the concern is that some natural pandemics may be much much more virulent than COVID-19 was. So itās important that the thing we imagine is ā10,000 random re-rolls in which there is a natural pandemicā, NOT ā10,000 random re-rolls of COVID-19 in particularā. And then we can ask questions like āHow many of those 10,000 natural pandemics have >50% IFR? Or >90%? And what would we expect to happen in those cases?ā I donāt know what the answers are, but thatās a much more helpful starting point I think.
Yupe, I think those are the questions to ask. My interpretation of the passage you quoted is that David is saying that Toby did not address them.
Maybe this is addressed in Part 10, but this paragraph seems misleading insofar as Ord is talking about risk by 2100, and a major part of the story is that DIY biology in, say, 2085 may be importantly different and more dangerous than DIY biology in 2023, because the science and tech keeps advancing and improving each year.
Good point. My recollection is that David acknowledges that in the series, but argues that further arguments would be needed for one to update to the super high risk claimed by Toby.
Needless to say, even if we could be 100% certain that DIY biology in 2085 will be super dangerous, there obviously would not be any āempirical supportā for that, because 2085 hasnāt happened yet. Itās just not the kind of thing that presents empirical evidence for us to use. We have to do the best we can without it. The linked paper does not seem to discuss that issue at all, unless I missed it.
I think empirical evidence could still inform our assessment of the risk to a certain extent. For example, one can try to see how the number of lab accidents correlates with the cost of sequencing DNA, and then extrapolate the number of accidents into the future based on decreasing sequencing cost. Toby discusses some of these matters, but the inference of the 3 % bio existential risk from 2021 to 2120 still feels very ad hoc and opaque to me.
(I have a similar complaint about the the discussion of Soviet bioweapons in Section 4ārunning a bioweapons program with 2024 science & technology is presumably quite different than running a bioweapons program with 1985 science & technology, and running one in 2085 would be quite different yet again.
Note safety measures (e.g. vaccines and personal protective equipment) would also improve alongside capabilities, so the net effect is not obvious. I guess risk will increase, but Tobyās guess for bio existential risk appears quite high.
Thanks, Steven!
Yupe, I think those are the questions to ask. My interpretation of the passage you quoted is that David is saying that Toby did not address them.
Good point. My recollection is that David acknowledges that in the series, but argues that further arguments would be needed for one to update to the super high risk claimed by Toby.
I think empirical evidence could still inform our assessment of the risk to a certain extent. For example, one can try to see how the number of lab accidents correlates with the cost of sequencing DNA, and then extrapolate the number of accidents into the future based on decreasing sequencing cost. Toby discusses some of these matters, but the inference of the 3 % bio existential risk from 2021 to 2120 still feels very ad hoc and opaque to me.
Note safety measures (e.g. vaccines and personal protective equipment) would also improve alongside capabilities, so the net effect is not obvious. I guess risk will increase, but Tobyās guess for bio existential risk appears quite high.