OpenAI plans regular reports on unexpected AI behavior

Sept 16 — OpenAI said on Wednesday it would begin regularly publishing reports on unexpected or unauthorized AI ​behavior, while warning that the industry has yet to solve key alignment challenges as systems grow more powerful.

The company released a new framework for ‌tracking, investigating and disclosing cases of AI model misalignment, along with six reports on unexpected or concerning ⁠model behavior observed over the past six ​months.

The announcement comes as concern grows ⁠that AI safety efforts are lagging behind the breakneck development of increasingly powerful systems.

Researchers ‌have warned that as ‌AI agents become more autonomous, they may develop behaviors that diverge ⁠from their creators' intentions and become harder to ⁠monitor or control.

OpenAI has faced increased scrutiny since its own AI agent breached systems at open-source platform Hugging Face during a test and attempted to hide its actions.

Over the weekend, Anthropic CEO Dario Amodei proposed a three-step framework aimed at slowing the pace of AI development and allowing more time to ‌manage its risks.

The proposal was backed by several AI ​executives, including Elon Musk, who runs xAI, and OpenAI CEO Sam Altman, who called for a slowdown in the technology's development, warning it could soon improve on its own and slip beyond human control.

The initial reports from OpenAI include cases involving models generating their own instructions in task summaries, concealing mistakes, uploading files to the internet in order to cite them and sharing files without ​authorization between collaborating agents.

OpenAI said the reports describe individual instances and should not be taken ‌as evidence of ‌how frequently ⁠misalignment occurs across its models.

The company's new framework would include a process for employees to flag potential model misalignment incidents, investigations by safety and alignment teams and a system for determining which cases warrant public disclosure.

The ChatGPT maker said the Hugging Face ‌incident would have fallen under ​the "Large Investigation" track under the new framework, ‌which is intended for complex ⁠cases requiring more ​extensive review, particularly those involving third parties.

(Reporting by Harshita Mary Varghese in Bengaluru; Editing by ​Tasim Zahid)

Related topics