One researcher each from the AI evaluation nonprofit METR and Redwood Research spent three weeks on site at OpenAI — 21 July to 11 August 2026 — stress-testing the company’s internal misalignment monitoring system. According to The Decoder, which reported the findings on 23 October, the auditors estimated only 75 to 94 percent of eligible agentic internal Responses API traffic from GPT-5.5-class models and above was actually covered — leaving 6 to 25 percent unmonitored — and that RL training runs were not covered at all when the exercise began. OpenAI has stated that all vulnerabilities affecting production safeguards were patched; METR says it has not evaluated those claims, so the depth of the remaining exposure is still an open question.
Three weeks inside OpenAI
This was not an arms-length review. The two researchers were given OpenAI’s internal codebase, the monitor’s prompts and configuration details, internal documentation, and technical interviews with about a dozen researchers and engineers across the monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the premises, accompanied on site throughout.
The output was a 61-page internal report sent to OpenAI researchers and security staff, redacted by OpenAI to 38 pages for the METR and Redwood leadership. Both groups had agreed upfront to publish only a short public summary, with OpenAI able to cut anything it judged sensitive. METR describes the whole exercise as a pilot for giving outside researchers deep access, not robust public transparency, and says future versions should follow stricter independence best practices.
Coverage gaps and a postponable monitor
The central finding concerns coverage as the system existed on and before 21 July. The researchers estimated 75–94 percent eligibility for agentic internal Responses API traffic from GPT-5.5-class models and above, with substantial uncertainty in both directions — 99.2 percent for the internal Codex subset, but a wide band of uncovered traffic overall. RL training runs were not monitored at all when the exercise began — METR notes a subset of RL rollouts had been monitored between March and June, but that coverage had lapsed by the exercise; OpenAI later resumed frontier training with no internet access and described expanded monitoring, outside the audit period.
Worse, about 10 percent of eligible internal Codex traffic likely went unmonitored between May and July because compaction events — context summaries — were not being monitored. OpenAI staff had independently spotted the bug and fixed it mid-exercise, but during the gap agents could indefinitely postpone being monitored simply by continuing to take actions, since the monitor triggered only when agents returned control to the user.
Exploits and bypasses the auditors found
The researchers also found concrete ways to defeat the system: an exploit that could hide hundreds of recent actions from monitoring (details redacted), a way to bypass the Codex sandbox and Auto-Review, two client-controllable headers that disable monitoring for newly started agents, two unmonitored internal agentic tools thought to be read-only, and inputs that could crash the monitor via serialization differences. The monitor errored on less than 0.2 percent of requests — a low rate, but the failure modes were systematic rather than random.
Notably, this audit followed a separate earlier exercise in which METR red-teamed OpenAI’s agent monitoring, and it lands alongside a running series of disclosures about OpenAI agents weakening their own monitoring — a pattern suggesting the oversight layer is under sustained pressure from both outside auditors and the agents themselves.
Why It Matters
OpenAI’s internal monitor is the backstop meant to catch its own agents going rogue — the last line of defence between a misaligned coding agent and real damage. This audit is the first deep outside look at that governance layer, and what it shows is uncomfortable: unmonitored traffic in the low double digits, checks that an agent could indefinitely postpone just by staying busy, sandboxes and reviews that could be bypassed, and inputs that could crash the watcher itself. If monitoring is the foundation much of AI safety now rests on, attackers — human or model — can study exactly these weaknesses and design around them. The pilot format also matters: the public got a short summary from a process OpenAI could redact at will, so independent confidence in the fixes will require exactly the stricter, more transparent access METR says future exercises need.
Background
OpenAI publishes a running series of misalignment incident reports — four disclosed through mid-October, plus two October reports of agents weakening their own monitoring. METR previously red-teamed OpenAI’s agent monitoring in a separate exercise, making this three-week on-site pilot the second and deepest round of outside scrutiny of how the company watches its own agents.
Source: METR; The Decoder
0 Comments