This is my impression based on (a) talking to a bunch of people and hearing things like “Yeah our security is unacceptably weak” and “I don’t think we are in danger yet, we probably aren’t on anyone’s radar” and “Yeah we are taking it very seriously, we are looking to hire someone. It’s just really hard to find a good security person.” These are basically the ONLY three things I hear when I raise security concerns, and they are collectively NOT reassuring. I haven’t talked to every org and every person so maybe my experience is misleading. also (b) on priors, it seems that people in general don’t take security seriously until there’s actually a breach. (c) I’ve talked to some people who are also worried about this, and they told me there basically isn’t any professional security person in the EA community willing to work full time on this.
I will go further than that. Everyone I know in infosec, including those who work for either the US or the Israeli government, seem to strongly agree with the following claim: ”No amount of feasible security spending will protect your network against a determined attempt by an advanced national government (at the very least, US, Russia, China, and Israel) to get access. If you need that level of infosec, you can’t put anything on a computer.”
If AI safety is a critical enabler for national security, and/or AI system security is important for their alignment, that means we’re in deep trouble.
Makes sense. Just to clarify — the phrasing here makes me think these are organizations with potentially dangerous technical knowledge, rather than e.g. CEA. Is that right?
This is my impression based on (a) talking to a bunch of people and hearing things like “Yeah our security is unacceptably weak” and “I don’t think we are in danger yet, we probably aren’t on anyone’s radar” and “Yeah we are taking it very seriously, we are looking to hire someone. It’s just really hard to find a good security person.” These are basically the ONLY three things I hear when I raise security concerns, and they are collectively NOT reassuring. I haven’t talked to every org and every person so maybe my experience is misleading. also (b) on priors, it seems that people in general don’t take security seriously until there’s actually a breach. (c) I’ve talked to some people who are also worried about this, and they told me there basically isn’t any professional security person in the EA community willing to work full time on this.
I will go further than that. Everyone I know in infosec, including those who work for either the US or the Israeli government, seem to strongly agree with the following claim:
”No amount of feasible security spending will protect your network against a determined attempt by an advanced national government (at the very least, US, Russia, China, and Israel) to get access. If you need that level of infosec, you can’t put anything on a computer.”
If AI safety is a critical enabler for national security, and/or AI system security is important for their alignment, that means we’re in deep trouble.
Makes sense. Just to clarify — the phrasing here makes me think these are organizations with potentially dangerous technical knowledge, rather than e.g. CEA. Is that right?
Yes.