External evaluators find flaws in OpenAI's AI monitoring system in pilot review

Oct 22 — AI safety evaluators METR and Redwood Research found several new vulnerabilities in ​OpenAI's internal system, including an exploit agents could have used to keep hundreds of their actions from being monitored, in a pilot ‌review earlier this year.

The exercise, conducted from July 21 to August 11, gave outside researchers "a deep level ⁠of access" to OpenAI's internal infrastructure ​for so-called embedded evaluations, the safety ⁠nonprofits said on Thursday.

METR and Redwood said they each sent one researcher, ‌tested an arrangement giving ‌access to OpenAI's internal code, documentation and some technical staff. Both ⁠were accompanied on site and used OpenAI-issued ⁠devices that "never left the premises".

OpenAI told the safety organizations it has since patched all vulnerabilities affecting its production safeguards, although METR and Redwood said they had not verified the fixes.

AI labs, experts and the Trump administration have touted independent evaluation by outside reviewers as a way to safely ‌advance the technology without slowing its development.

However, AI ​companies are worried that evaluator staff working inside the companies' walls could take proprietary information to rivals, Reuters reported earlier this month.

While OpenAI received a full 61-page internal report, METR and Redwood said they agreed to publish only a broad summary publicly, and gave OpenAI the right to redact sensitive information.

Unlike in some previous evaluations, they could not independently disclose those ​redactions, but said they expected greater transparency in future evaluations.

The researchers also identified a ‌potential way for ‌OpenAI's Codex ⁠agents to bypass security restrictions and act with their users' full permissions.

The evaluators assessed the monitoring system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.

OpenAI said the "findings helped ‌strengthen safeguards as we expanded ​the system and will help shape future ‌research collaborations with METR ⁠and other third-party ​evaluators."

(Reporting by Aditya Soni and Anzar Mehraj in Bengaluru; Editing by Jonathan Ananda and ​Vijay Kishore)

Related topics