Overall I think this post was well done and introduces valuable approaches, especially the focus on social engineering and the limits of psychometric data collection available to most firms, but generally a lot of different and valuable topics which I expect to be unfamiliar to most readers. Have you thought about cross posting this to Lesswrong, where users are friendlier to AI safety?
Something that’s worth keeping in mind is that, although base rates of this seem low relative to employee incompetence, it’s also true that a sufficiently sophisticated adversary will be highly capable of framing specific employees for the attack. This is important for AI labs, which will be noticed by unusually powerful adversaries, and yet nonetheless must get everything right. They should expect their top-performing security staff to start getting bumped off one by one, the same way they would expect that to happen to leadership.
Train employees on personal cybersecurity (securing smart home devices, managing logins, financial scams, etc.).
Securing smart home devices isn’t possible, not for anyone in or adjacent to EA anyway. The idea that you can make smart devices themselves safe is ludicrous and dangerous; unless by “securing smart home devices” you meant mitigating personal exposure to them, which I absolutely agree would reduce risk. The threat of smart home devices creating massive psychometric datasets and researching social engineering with sample sizes in the millions is a security nightmare, that every developed country has gotten entangled in; just because everyone’s doing it doesn’t make it sensible or reasonable, just like religion or meat consumption. The current paradigm of constant smart device exposure is already wildly inadequate for the basic infosec that AI labs currently require, let alone for the transformative slow takeoff world that many anticipate over the next 1-2 decades.
I’ve crossposted it now. If there are other forums relevant to cybersecurity topics in EA in particular, I’d appreciate suggestions :-)
RE: Personal Cybersecurity and IoT
Yes, I agree that the best way to improve cybersecurity with personal IoT devices is to avoid them. I’ll update the wording to be more clear about that.
Overall I think this post was well done and introduces valuable approaches, especially the focus on social engineering and the limits of psychometric data collection available to most firms, but generally a lot of different and valuable topics which I expect to be unfamiliar to most readers. Have you thought about cross posting this to Lesswrong, where users are friendlier to AI safety?
Something that’s worth keeping in mind is that, although base rates of this seem low relative to employee incompetence, it’s also true that a sufficiently sophisticated adversary will be highly capable of framing specific employees for the attack. This is important for AI labs, which will be noticed by unusually powerful adversaries, and yet nonetheless must get everything right. They should expect their top-performing security staff to start getting bumped off one by one, the same way they would expect that to happen to leadership.
Securing smart home devices isn’t possible, not for anyone in or adjacent to EA anyway. The idea that you can make smart devices themselves safe is ludicrous and dangerous; unless by “securing smart home devices” you meant mitigating personal exposure to them, which I absolutely agree would reduce risk. The threat of smart home devices creating massive psychometric datasets and researching social engineering with sample sizes in the millions is a security nightmare, that every developed country has gotten entangled in; just because everyone’s doing it doesn’t make it sensible or reasonable, just like religion or meat consumption. The current paradigm of constant smart device exposure is already wildly inadequate for the basic infosec that AI labs currently require, let alone for the transformative slow takeoff world that many anticipate over the next 1-2 decades.
@trevor1 Thank you for the detailed response!
RE: Crossposting to LessWrong
I’ve crossposted it now. If there are other forums relevant to cybersecurity topics in EA in particular, I’d appreciate suggestions :-)
RE: Personal Cybersecurity and IoT
Yes, I agree that the best way to improve cybersecurity with personal IoT devices is to avoid them. I’ll update the wording to be more clear about that.