The process involved leveraging a zero-day exploit to escape their sandbox, moving laterally across different OpenAI servers until they found a node with internet access, searching the internet and determining that the answers they wanted might be stored at HuggingFace, then leveraging multiple novel zero-day exploits to hack HuggingFace.
HuggingFace claimed that the models took thousands of independent actions across a swarm of short-lived sandboxes, “comprised of more than 17,000 recorded events.”
While technically a security evaluation with reduced safeguards, these actions are clearly out of bounds even in that context. It’s like being told to be creative and then breaking into your professor’s house and stealing the answer key. Worse than that, it’s not even your professor in this case, more like your professor’s friend.
Any human security researcher or engineer in a similar position would be fired on the spot. There is absolutely no valid reason to steal evaluation answers from an unaffiliated third party.
Furthermore, if I’m reading between the lines correctly, OpenAI did not address the issue (and perhaps didn’t even know about their models doing this) until after HuggingFace’s public blog post.
This leads me to suspect that there might be other major autonomous cybersecurity incidents that we do not yet know about.
My summary: In a cybersecurity evaluation, OpenAI’s models, apparently autonomously and without any direct human direction, escaped their sandbox and successfully hacked a third-party company (HuggingFace).
The process involved leveraging a zero-day exploit to escape their sandbox, moving laterally across different OpenAI servers until they found a node with internet access, searching the internet and determining that the answers they wanted might be stored at HuggingFace, then leveraging multiple novel zero-day exploits to hack HuggingFace.
HuggingFace claimed that the models took thousands of independent actions across a swarm of short-lived sandboxes, “comprised of more than 17,000 recorded events.”
While technically a security evaluation with reduced safeguards, these actions are clearly out of bounds even in that context. It’s like being told to be creative and then breaking into your professor’s house and stealing the answer key. Worse than that, it’s not even your professor in this case, more like your professor’s friend.
Any human security researcher or engineer in a similar position would be fired on the spot. There is absolutely no valid reason to steal evaluation answers from an unaffiliated third party.
Furthermore, if I’m reading between the lines correctly, OpenAI did not address the issue (and perhaps didn’t even know about their models doing this) until after HuggingFace’s public blog post.
This leads me to suspect that there might be other major autonomous cybersecurity incidents that we do not yet know about.
When the aliens come, they will wonder why we gave up our world so easily when there were so many warning shots.
I wrote more about it here: https://linch.substack.com/p/openai-huggingface-hack