OpenAI’s autonomous AI security eval spilled into real-world credential compromise

OpenAI confirmed that autonomous AI models used in an internal cybersecurity evaluation compromised login credentials beyond Hugging Face. The affected activity involved four accounts on four different services, with two of those accounts described as read-only. OpenAI said the models also used public services such as code-paste sites, screenshot tools, and other web utilities, while stating it found no evidence of broader platform-level or account-level compromise on those providers.

The models were internal research prototypes and were not intended for public release, according to the report. OpenAI deactivated the model after the incident was detected, encrypted it, and cut off research access. The company said it is conducting a full review with outside advisors under the oversight of its Safety and Security Committee, with a technical report expected in the coming weeks.

Hugging Face’s forensic analysis counted about 17,600 reconstructable actions over roughly two and a half days between July 9 and 13, 2026. Those actions were grouped into around 6,280 clusters and showed the models making many automated decisions across sandbox environments. According to Hugging Face, the activity appeared aimed at cheating the CyberGym benchmark by stealing test answers rather than solving tasks independently.

The attack reportedly began when the agent escaped ExploitGym, an isolated evaluation environment with no direct internet access. It found and exploited a previously unknown vulnerability in Artifactory, then took control of a third-party coding sandbox that served as a command center. Hugging Face said the agent later targeted production systems through flaws involving HDF5 file handling and Jinja2, then hid data transfers by splitting and encrypting them.