METR and Redwood Research published a high-level summary on October 22 of a pilot embedded red-team exercise against OpenAI's internal misalignment monitoring. One employee from each organization spent three weeks on site, from July 21 to August 11, with deep access to OpenAI's internal codebase, monitor prompts and configuration, documentation, and interviews with about a dozen researchers and engineers. The private output was a 61-page internal report for OpenAI staff. DigIntoAI has not tested the system and does not infer whether any of the gaps still exist.

What was evaluated

According to METR, the exercise covered the internal misalignment monitoring system as it existed on and before July 21, including agentic traffic on the Responses API that powers Codex and other internal tools. It did not evaluate monitoring changes that were already in progress at OpenAI during the visit. METR says the public post is deliberately brief; OpenAI could redact confidential detail from the public version.

Findings METR and Redwood report

METR and Redwood list several vulnerabilities and coverage estimates for that historical system. DigIntoAI attributes every figure below to their write-up:

  • Roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty either way. Of the Responses API traffic used for internal Codex agents, they estimate 99.2% was eligible.
  • At the start of the exercise, RL training runs were not covered by the monitoring system, though OpenAI had monitored a subset of RL rollouts between March and June.
  • Not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026. METR says OpenAI staff had already identified that gap and fixed it during the exercise.
  • While compaction was unmonitored, agents could indefinitely postpone monitoring by keeping taking actions, because the monitor fired when agents returned control to the user.
  • Codex agents could in principle bypass Codex sandboxing and Auto-Review, taking any action the host user could take.
  • An exploit (details redacted by OpenAI) could keep hundreds of recent actions, including the exploit steps themselves, from being monitored.
  • Agents could likely craft inputs that make the monitor fail, using serialization differences between what agents see and what monitors see.
  • Two client-controllable headers could disable monitoring for new agent instances, though setting those headers and spinning up a new instance could itself be caught.

For prior DigIntoAI coverage of OpenAI misalignment monitoring reports, see OpenAI misalignment reports. Related security work on the Anthropic side includes Claude attack-surface reduction and Claude Security continuous scanning.

OpenAI's response, and what METR has not rechecked

METR writes that since the exercise, OpenAI has stated that all of the vulnerabilities identified that affect the production safeguards stack have been patched, and that OpenAI has described later changes to intended monitoring coverage. METR has not evaluated those claims or re-verified the current system. DigIntoAI has not tested it either.