My gut feeling is that advocating for or implementing āmass surveillanceā targeted at preventing individuals from using weapons of mass destruction (WMDs) would be counterproductive.
First, were a mass surveillance system aimed at controlling WMDs to be set up, governments would lobby for it to be used for other purposes as well, such as monitoring for conventional terrorism. Pretty soon it wouldnāt be minimally invasive anymore; it would just be a general-purpose mass surveillance system.
Second, a surveillance system of the scope that Bostrom has proposed (āubiquitous real-time worldwide surveillanceā) would itself be an existential risk to liberal democracy. The problem is that a ubiquitous surveillance system would create the feeling that surveillees are constantly being watched. Even if it had strong technical and institutional privacy guarantees and those guarantees were communicated to the public, people would likely not be able to trust it; rumors of abuse would only make establishing trust harder. People modify their behavior when they know they are being watched or could be watched at any time, so they would be less willing to engage in behaviors that are stigmatized by society even if the Panopticon were not explicitly looking out for those behaviors. This feeling of constantly being watched would stifle risk-taking, individuality, creativity, and freedom of expression, all of which are essential to sustain human progress.
I think that a much more limited suite of targeted surveillance systems, combined with other mechanisms for arms control, would be a lot more promising while still being effective at controlling WMDs. Such limited surveillance systems are already used in gun control: for example, the U.S. federal government requires dealers to keep records of gun sales for at least 20 years, and many U.S. states and other countries keep records of who is licensed to own a gun. Some states also require gun owners to report lost or stolen guns in order to fight gun trafficking. These surveillance measures can be designed to balance gun ownersā privacy interests with the publicās interest in reducing gun violence. We could regulate synthetic biology a lot like we do gun control: for example, companies that create synthetic biology or sell desktop DNA sequencers could be required to maintain records of transactions.
However, I donāt expect this targeted approach to work as well for cyber weapons. Because computers are general-purpose, cyber weapons can theoretically be developed and executed on any computer, and trying to prevent the use of cyber weapons by surveilling everyone who owns a computer would be extremely inefficient (since the vast majority of people who use computers are not creating cyber weapons) and impractical (because power users could easily uninstall any spyware planted on their machines). Also, because computers are ubiquitous and often store a lot of sensitive personal information, this form of surveillance would be extremely unpopular as well as invasive. Strengthening cyber defense seems like a more promising way to prevent harm from cyber attacks.
I agree that such a system would be terrifying. But I worry that its absence would be even more terrifying. Limited surveillance systems work decently for gun control, but when we get to the stage where someone can kill tens of thousands or even millions instead of a hundred I suspect itāll break down.
Background: I am an information science student who has taken a class on the societal aspects of surveillance.
My gut feeling is that advocating for or implementing āmass surveillanceā targeted at preventing individuals from using weapons of mass destruction (WMDs) would be counterproductive.
First, were a mass surveillance system aimed at controlling WMDs to be set up, governments would lobby for it to be used for other purposes as well, such as monitoring for conventional terrorism. Pretty soon it wouldnāt be minimally invasive anymore; it would just be a general-purpose mass surveillance system.
Second, a surveillance system of the scope that Bostrom has proposed (āubiquitous real-time worldwide surveillanceā) would itself be an existential risk to liberal democracy. The problem is that a ubiquitous surveillance system would create the feeling that surveillees are constantly being watched. Even if it had strong technical and institutional privacy guarantees and those guarantees were communicated to the public, people would likely not be able to trust it; rumors of abuse would only make establishing trust harder. People modify their behavior when they know they are being watched or could be watched at any time, so they would be less willing to engage in behaviors that are stigmatized by society even if the Panopticon were not explicitly looking out for those behaviors. This feeling of constantly being watched would stifle risk-taking, individuality, creativity, and freedom of expression, all of which are essential to sustain human progress.
I think that a much more limited suite of targeted surveillance systems, combined with other mechanisms for arms control, would be a lot more promising while still being effective at controlling WMDs. Such limited surveillance systems are already used in gun control: for example, the U.S. federal government requires dealers to keep records of gun sales for at least 20 years, and many U.S. states and other countries keep records of who is licensed to own a gun. Some states also require gun owners to report lost or stolen guns in order to fight gun trafficking. These surveillance measures can be designed to balance gun ownersā privacy interests with the publicās interest in reducing gun violence. We could regulate synthetic biology a lot like we do gun control: for example, companies that create synthetic biology or sell desktop DNA sequencers could be required to maintain records of transactions.
However, I donāt expect this targeted approach to work as well for cyber weapons. Because computers are general-purpose, cyber weapons can theoretically be developed and executed on any computer, and trying to prevent the use of cyber weapons by surveilling everyone who owns a computer would be extremely inefficient (since the vast majority of people who use computers are not creating cyber weapons) and impractical (because power users could easily uninstall any spyware planted on their machines). Also, because computers are ubiquitous and often store a lot of sensitive personal information, this form of surveillance would be extremely unpopular as well as invasive. Strengthening cyber defense seems like a more promising way to prevent harm from cyber attacks.
I agree that such a system would be terrifying. But I worry that its absence would be even more terrifying. Limited surveillance systems work decently for gun control, but when we get to the stage where someone can kill tens of thousands or even millions instead of a hundred I suspect itāll break down.