Consider the following argument:
1) Over time humanity will discover more superweapons. At the moment, these are mostly just accessible to state actors, but eventually these will become accessible to smaller groups/individuals
2) The (edit: potential for unilateral action) means that if a large number of groups gain access such an event is almost guaranteed to occur
3) It seems unrealistic to believe that we could ever completely prevent such terrorism occurring without minimally-invasive mass surveillance. I don’t believe that we could obtain this result via education without it being in effect brainwashing. Maybe you could genetically engineer people to be less violent, but fundamentally changing our psychology is terrifying as well.
4) Minimally-invasive mass surveillance would purely focus on threats above a particular scale and ignore everything else that we more minor. Given sufficiently advanced technology, we might be able to prevent humans from having access to the information in any other circumstance
5) While it is possible that a superintelligence might be able to talk everyone into accepting that this is a reasonable policy, I am unsure enough about this claim to believe that it is worthwhile trying to build support for minimally-invasive mass surveillance as this will undoubtedly be reflexively opposed by many people who don’t appreciate the stakes.
It’s likely that I have seen this term mentioned somewhere else in the past by someone else, but if I did, the source is long gone from my memory.
What do you think about this argument?
Update: I was linked to this TED talk by Nick Bostrom where he discusses the potential that we might need such surveillance.