This seems like a thorough consideration of the interaction of BCIs with the risk of totalitarianism. I was also prompted to think a bit about BCIs as a GCR risk factor recently and had started compiling some references, but I haven’t yet refined my views as much as this.
One comment I have is that risk described here seems to rely not just on the development of any type of BCI but on a specific kind, namely, relatively cheap consumer BCIs that can nonetheless provide a high-fidelity bidirectional neural interface. It seems likely that this type of BCI would need to be invasive, but it’s not obvious to me that invasive BCI technology will inevitably progress in that direction. Musk hint’s that Neuralink’s goals are mass-market, but I expect that regulatory efforts could limit invasive BCI technology to medical use cases, and likewise, any military development of invasive BCI seems likely to lead to equipment that is too expensive for mass adoption (although it could provide the starting point for commercialization). Although DARPA’s Next-GenerationNonsurgicalNeurotechnology (N3) program does have the goal of developing high-fidelity non- or minimally-invasive BCIs; my intuition is at that they will not achieve their goal of reading from one million and writing to 100,000 neurons non-invasively, but I’m not sure about the potential of the minimally-invasive path. So one theoretical consideration is what percentage of a population needs to be thought policed to retain effective authoritarian control, which would then indicate how commercialized BCI technology would need to be before it could become a risk factor.
In my view, a reasonable way to steer BCIs development away from posing a risk-factor for totalitarianism would be to encourage the development of high-fidelity non-invasive and read-focused consumer BCI. While non-invasive devices are intrinsically more limited than invasive ones, if consumers can still be satisfied by their performance then it will reduce the demand to develop invasive technology. Facebook and Kernel already look like they are moving towards non-invasive technology. One company that I think is generally overlooked is CTRL-Labs (now owned by Facebook), who are developing an armband that acquires high-fidelity measurements from motor neurons—although this is a peripheral nervous system recording, users can apparently repurpose motor neurons for different tasks and even learn to control the activity of individual neurons (see this promotional video). As an aside, if anybody is interested in working on non-invasive BCI hardware, I have a project proposal for developing a device for acquiring high-fidelity and non-invasive central nervous system activity measurements that I’m no longer planning to pursue but am able to share.
The idea of BCIs that punish dissenting thoughts being used to condition people away from even thinking about dissent may have a potential loophole, in that such conditioning could lead people to avoid thinking such thoughts or it could simply lead them to think such thoughts in ways that aren’t punished. I expect that human brains have sufficient plasticity to be able to accomplish this under some circumstances and while the punishment controller could also adapt what it punishes to try and catch such evasive thoughts, it may not always have an advantage and I don’t think BCI thought policing could be assumed to be 100% effective. More broadly, differences in both intra- or inter-person thought patterns could determine how effective BCI is for thought policing. If a BCI monitoring algorithm can be developed using a small pool of subjects and then applied en masse, that seems much risky than if the monitoring algorithm needs to be adapted to each individual and possibly updated over time (though there would be scope for automating updating). I expect that Neuralinks future work will indicate how ‘portable’ neural decoding and encoding algorithms are between individuals.
I have a fun anecdotal example of neural activity diversity: when I was doing my PhD at the Queensland Brain Institute I did a pilot experiment for an fMRI study on visual navigation for a colleague’s experiment. Afterwards, he said that my neural responses were quite different from those of the other pilot participant (we both did the navigation task well). He completed and published the study and ask the other pilot participant to join other fMRI experiments he ran, but never asked me to participate again. I’ve wondered if I was the one who ended up having the weird neural response compared to the rest of the participants in that study… (although my structural MRI scans are normal, so it’s not like I have a completely wacky brain!)
The BCI risk scenario I’ve considered is whether BCIs could provide a disruptive improvement in a user’s computer-interface speed or another cognitive domain. DARPA’s Neurotechnology for Intelligence Analysts (NIA) program showed that an x10 increase in image analysis speed with no loss of accuracy, just using EEG (see here for a good summary of DARPAs BCI programs until 2015). It seems reasonable that somewhat larger speed improvements could be attained using invasive BCI, and this speed improvement would probably generalize to other, more complicated tasks. When advanced BCIs is limited to early adopters, could such cognitive advantages facilitate the risky development in AI or bioweapons by small teams, or give operational advantages to intelligence agencies or militaries? (happy to discuss or share my notes on this with anybody who is interested in looking into this aspect further)
This seems like a thorough consideration of the interaction of BCIs with the risk of totalitarianism. I was also prompted to think a bit about BCIs as a GCR risk factor recently and had started compiling some references, but I haven’t yet refined my views as much as this.
One comment I have is that risk described here seems to rely not just on the development of any type of BCI but on a specific kind, namely, relatively cheap consumer BCIs that can nonetheless provide a high-fidelity bidirectional neural interface. It seems likely that this type of BCI would need to be invasive, but it’s not obvious to me that invasive BCI technology will inevitably progress in that direction. Musk hint’s that Neuralink’s goals are mass-market, but I expect that regulatory efforts could limit invasive BCI technology to medical use cases, and likewise, any military development of invasive BCI seems likely to lead to equipment that is too expensive for mass adoption (although it could provide the starting point for commercialization). Although DARPA’s Next-Generation Nonsurgical Neurotechnology (N3) program does have the goal of developing high-fidelity non- or minimally-invasive BCIs; my intuition is at that they will not achieve their goal of reading from one million and writing to 100,000 neurons non-invasively, but I’m not sure about the potential of the minimally-invasive path. So one theoretical consideration is what percentage of a population needs to be thought policed to retain effective authoritarian control, which would then indicate how commercialized BCI technology would need to be before it could become a risk factor.
In my view, a reasonable way to steer BCIs development away from posing a risk-factor for totalitarianism would be to encourage the development of high-fidelity non-invasive and read-focused consumer BCI. While non-invasive devices are intrinsically more limited than invasive ones, if consumers can still be satisfied by their performance then it will reduce the demand to develop invasive technology. Facebook and Kernel already look like they are moving towards non-invasive technology. One company that I think is generally overlooked is CTRL-Labs (now owned by Facebook), who are developing an armband that acquires high-fidelity measurements from motor neurons—although this is a peripheral nervous system recording, users can apparently repurpose motor neurons for different tasks and even learn to control the activity of individual neurons (see this promotional video). As an aside, if anybody is interested in working on non-invasive BCI hardware, I have a project proposal for developing a device for acquiring high-fidelity and non-invasive central nervous system activity measurements that I’m no longer planning to pursue but am able to share.
The idea of BCIs that punish dissenting thoughts being used to condition people away from even thinking about dissent may have a potential loophole, in that such conditioning could lead people to avoid thinking such thoughts or it could simply lead them to think such thoughts in ways that aren’t punished. I expect that human brains have sufficient plasticity to be able to accomplish this under some circumstances and while the punishment controller could also adapt what it punishes to try and catch such evasive thoughts, it may not always have an advantage and I don’t think BCI thought policing could be assumed to be 100% effective. More broadly, differences in both intra- or inter-person thought patterns could determine how effective BCI is for thought policing. If a BCI monitoring algorithm can be developed using a small pool of subjects and then applied en masse, that seems much risky than if the monitoring algorithm needs to be adapted to each individual and possibly updated over time (though there would be scope for automating updating). I expect that Neuralinks future work will indicate how ‘portable’ neural decoding and encoding algorithms are between individuals.
I have a fun anecdotal example of neural activity diversity: when I was doing my PhD at the Queensland Brain Institute I did a pilot experiment for an fMRI study on visual navigation for a colleague’s experiment. Afterwards, he said that my neural responses were quite different from those of the other pilot participant (we both did the navigation task well). He completed and published the study and ask the other pilot participant to join other fMRI experiments he ran, but never asked me to participate again. I’ve wondered if I was the one who ended up having the weird neural response compared to the rest of the participants in that study… (although my structural MRI scans are normal, so it’s not like I have a completely wacky brain!)
The BCI risk scenario I’ve considered is whether BCIs could provide a disruptive improvement in a user’s computer-interface speed or another cognitive domain. DARPA’s Neurotechnology for Intelligence Analysts (NIA) program showed that an x10 increase in image analysis speed with no loss of accuracy, just using EEG (see here for a good summary of DARPAs BCI programs until 2015). It seems reasonable that somewhat larger speed improvements could be attained using invasive BCI, and this speed improvement would probably generalize to other, more complicated tasks. When advanced BCIs is limited to early adopters, could such cognitive advantages facilitate the risky development in AI or bioweapons by small teams, or give operational advantages to intelligence agencies or militaries? (happy to discuss or share my notes on this with anybody who is interested in looking into this aspect further)