Bostrom [claims] that even a small credence in the VWH requires continuously controlling and surveilling everyone on earth.
Bostrom does not claim this. Period. (Your reading is good-faith, but Bostrom is frequently misread on this topic by people criticizing him in bad faith, so it’s worth emphasizing—and it’s just an important point.) He narrowly claims here that mass surveillance would be necessary given “a biotechnological black ball that is powerful enough that a single malicious use could cause a pandemic that would kill billions of people.”
Another relevant Bostrom quote:
Comprehensive surveillance and global governance would thus offer protection against a wide spectrum of civilizational vulnerabilities. This is a considerable reason in favor of bringing about those conditions. The strength of this reason is roughly proportional to the probability that the vulnerable world hypothesis is true.
He goes on to discuss the downsides of surveillance and global governance. So your quotes like “Bostrom’s plan [purports to be a] one-size-fits-all antidote” are not correct, and Bostrom would agree with you that totalitarianism and surveillance present astronomical risks.
This is fair. I got a little sloppy with my language there, but elsewhere I note that Bostrom is arguing for this state “pro-tanto” not “all things considered.” My reading is that the panopticon proposal is mostly a rhetorical strategy to give a concrete image of what the massive expected values of existential risk might justify.
I still think that his narrow claim is wrong though. “Mass surveillance would be necessary given “a biotechnological black ball that is powerful enough that a single malicious use could cause a pandemic that would kill billions of people.”
Even in the conditional world where the deadly pandemic exists, mass surveillance is only good if it is used to actually stop the pandemic and does not cause more harm afterwards. I don’t think either of these things are very likely if they’re attached to any form of government we’re familiar with. Mass surveillance isn’t even necessary, it’s just one possible technological solution to a bio-tech black ball. Really good vaccines or PPE or genetic improvements to the immune system would also suffice.
I disagree that you just “got a little sloppy”; you exaggerate Bostrom’s policy recommendations elsewhere too and generally frame the relevant parts of your piece as arguing against Bostrom rather than as arguing against someone who advocates positions that Bostrom merely analyzes. Most readers would get the sense that “Bostrom claims that his global surveillance solution to anthropogenic risks is a one-size-fits all antidote”; this is false.
And of course I agree—and Bostrom would agree—that there are many possible solutions and countermeasures to dangerous biotechnology. But if we’re assuming that a particular technology is a black ball presenting a type-1 vulnerability, as Bostrom does for the sake of illustration in one paragraph, we are necessarily assuming that (1) it devastates civilization by default, so we are necessarily assuming that eg PPE won’t save us, and (2) it is available to a large number of actors by default, so we need something like mass surveillance to preempt use. So I think you’re saying something reasonable, but not really disagreeing with Bostrom.
“Some areas, such as synthetic biology, could produce a discovery that suddenly democratizes mass destruction, e.g. by empowering individuals to kill hundreds of millions of people using readily available materials. In order for civilization to have a general capacity to deal with “black ball” inventions of this type, it would need a system of ubiquitous real-time worldwide surveillance. In some scenarios, such a system would need to be in place before the technology is invented.”
So if we assume that some black balls like this are in the urn which I do in the essay, this is a position that Bostrom explicitly advocates, not just one which he analyzes. But even assuming that the VWH is true and a technology like this does exist, I don’t think this policy recommendation is helpful.
State enforced “ubiquitous real-time worldwide surveillance” is neither a necessary nor sufficient technology to address a type-1 vulnerability like this unless the definition of type-1 vulnerability trivially assumes that it is. Advanced technology that democratizes protection like vaccines, PPE, or drugs can alleviate a risk like this, so a panopticon is not necessary. A state with ubiquitous surveillance need not stop pandemics to stay rich and powerful and indeed may create them to keep their position.
Even if we knew a black ball was coming, setting up a panopticon would probably do more harm than good, and it certainly would if we didn’t come up with any new ways of aligning and constraining state power. I don’t think Bostrom would agree with that statement but that is what I defend in the essay. Do you think Bostrom would agree with that on your reading of the VWH?
Bostrom does not claim this. Period. (Your reading is good-faith, but Bostrom is frequently misread on this topic by people criticizing him in bad faith, so it’s worth emphasizing—and it’s just an important point.) He narrowly claims here that mass surveillance would be necessary given “a biotechnological black ball that is powerful enough that a single malicious use could cause a pandemic that would kill billions of people.”
Another relevant Bostrom quote:
He goes on to discuss the downsides of surveillance and global governance. So your quotes like “Bostrom’s plan [purports to be a] one-size-fits-all antidote” are not correct, and Bostrom would agree with you that totalitarianism and surveillance present astronomical risks.
This is fair. I got a little sloppy with my language there, but elsewhere I note that Bostrom is arguing for this state “pro-tanto” not “all things considered.” My reading is that the panopticon proposal is mostly a rhetorical strategy to give a concrete image of what the massive expected values of existential risk might justify.
I still think that his narrow claim is wrong though. “Mass surveillance would be necessary given “a biotechnological black ball that is powerful enough that a single malicious use could cause a pandemic that would kill billions of people.”
Even in the conditional world where the deadly pandemic exists, mass surveillance is only good if it is used to actually stop the pandemic and does not cause more harm afterwards. I don’t think either of these things are very likely if they’re attached to any form of government we’re familiar with. Mass surveillance isn’t even necessary, it’s just one possible technological solution to a bio-tech black ball. Really good vaccines or PPE or genetic improvements to the immune system would also suffice.
Thanks for your reply.
I disagree that you just “got a little sloppy”; you exaggerate Bostrom’s policy recommendations elsewhere too and generally frame the relevant parts of your piece as arguing against Bostrom rather than as arguing against someone who advocates positions that Bostrom merely analyzes. Most readers would get the sense that “Bostrom claims that his global surveillance solution to anthropogenic risks is a one-size-fits all antidote”; this is false.
And of course I agree—and Bostrom would agree—that there are many possible solutions and countermeasures to dangerous biotechnology. But if we’re assuming that a particular technology is a black ball presenting a type-1 vulnerability, as Bostrom does for the sake of illustration in one paragraph, we are necessarily assuming that (1) it devastates civilization by default, so we are necessarily assuming that eg PPE won’t save us, and (2) it is available to a large number of actors by default, so we need something like mass surveillance to preempt use. So I think you’re saying something reasonable, but not really disagreeing with Bostrom.
Bostrom says in the policy recommendations:
“Some areas, such as synthetic biology, could produce a discovery that suddenly democratizes mass destruction, e.g. by empowering individuals to kill hundreds of millions of people using readily available materials. In order for civilization to have a general capacity to deal with “black ball” inventions of this type, it would need a system of ubiquitous real-time worldwide surveillance. In some scenarios, such a system would need to be in place before the technology is invented.”
So if we assume that some black balls like this are in the urn which I do in the essay, this is a position that Bostrom explicitly advocates, not just one which he analyzes. But even assuming that the VWH is true and a technology like this does exist, I don’t think this policy recommendation is helpful.
State enforced “ubiquitous real-time worldwide surveillance” is neither a necessary nor sufficient technology to address a type-1 vulnerability like this unless the definition of type-1 vulnerability trivially assumes that it is. Advanced technology that democratizes protection like vaccines, PPE, or drugs can alleviate a risk like this, so a panopticon is not necessary. A state with ubiquitous surveillance need not stop pandemics to stay rich and powerful and indeed may create them to keep their position.
Even if we knew a black ball was coming, setting up a panopticon would probably do more harm than good, and it certainly would if we didn’t come up with any new ways of aligning and constraining state power. I don’t think Bostrom would agree with that statement but that is what I defend in the essay. Do you think Bostrom would agree with that on your reading of the VWH?