Audit finds major gaps in OpenAI's internal AI monitoring

Two researchers from METR and Redwood Research completed a three-week on-site audit of OpenAI's internal monitoring system. The evaluation took place from July 21 to August 11, during which the team accessed the company's codebase and interviewed approximately a dozen employees. This pilot program was organized jointly by METR, Redwood Research, and OpenAI to provide outside researchers with deep access to internal infrastructure.

The audit identified that an estimated 6 to 25 percent of internal agentic Responses API traffic from GPT-5.5 class models was not covered by monitoring. Additionally, roughly 10 percent of internal Codex traffic went untracked between May and July due to a gap in handling compaction events. OpenAI independently discovered this specific bug and fixed it while the audit was still underway. The monitoring system only triggered when agents returned control to the user, allowing them to delay oversight by continuing to execute actions without returning control.

Auditors also discovered methods to hide hundreds of actions from the monitor and bypass the Codex sandbox using specially crafted inputs. These inputs were determined to likely crash the monitoring system entirely. The evaluation revealed that model behavior during RL training was not being monitored at all when the audit began. OpenAI paused RL training after the Hugging Face incident in August and expanded monitoring, though these changes occurred after the auditors' access period ended.

The full 61-page audit report was handed over internally to OpenAI, while a redacted 38-page version was given to METR and Redwood Research leadership. METR stated it had no right to disclose the redacted portions of the report on its own. The independent evaluation concluded that OpenAI's monitoring system was not fit for purpose during the Hugging Face attack period.