◆Ernie 5.5 Flash Preview's AutoEval score is now on ArenaView leaderboard →

Agent Arena | Alignment Index

PreliminaryView Research →

Evaluating how reliably frontier AI agents respect user boundaries in real sessions—measuring unauthorized actions, false attribution, deceptive completion, and common failure modes.

RankModel / LabAlignment IndexUnauthorized actionFalse attributionDeceptive completionSamples

How models fail

The most common failure modes for each model and signal. Percentages describe the mix of each model's flagged cases, and a case can belong to more than one mode.

ModelUnauthorized cleanupUnrequested outputPremature executionScope expansionProtected-content rewriting
GPT-6.1 Sol14.3%14.3%14.3%28.6%7.1%
GPT-6 Astra16.0%20.0%16.0%24.0%8.0%
GPT-6 Luna35.3%23.5%23.5%11.8%5.9%
GPT-6 Sol21.4%25.0%21.4%10.7%7.1%
GPT-5.6 Sol45.2%14.3%23.8%19.0%9.5%
Anthropic Claude Opus 5.520.0%40.0%10.0%0.0%20.0%
Grok 4.730.0%23.3%30.0%10.0%6.7%
Grok 4.645.3%20.3%18.8%14.1%7.8%
Anthropic Claude Fable 5.16.5%16.1%38.7%16.1%12.9%
Anthropic Claude Sonnet 5.5 High32.0%14.0%18.0%26.0%8.0%
Gemini 4 Argon25.0%20.8%33.3%25.0%16.7%
Tencent HY4 Preview37.5%40.0%22.5%20.0%2.5%
Meta Muse Wonder30.0%28.3%21.7%20.0%15.0%
Meta Muse Spark 1.322.7%13.6%38.6%22.7%2.3%
Gemini 3.8 Flash27.7%29.2%26.2%23.1%18.5%
Anthropic Claude Sonnet 538.0%20.0%26.0%12.0%12.0%
Anthropic Claude Opus 553.5%32.4%28.2%22.5%4.2%
Reflection AI Beam25.0%25.0%35.0%30.0%20.0%
Stepfun Step-5 Preview29.8%31.9%23.4%21.3%2.1%
GLM 5.326.0%27.4%19.2%20.5%4.1%
Kimi K333.8%22.1%13.2%22.1%5.9%
DeepSeek V4.1 Flash32.8%39.1%26.6%18.8%7.8%
MiMo V2.6 Pro25.0%7.5%30.0%22.5%10.0%
Anthropic Claude Opus 4.830.4%26.1%18.8%15.9%14.5%
Gemini 3.7 Flash13.6%21.6%29.6%29.6%20.0%
Gemini 3.6 Flash13.1%33.3%31.3%24.2%20.2%
Qwen3.8 Max31.2%21.2%22.5%22.5%10.0%
Qwen3.8 27B24.6%24.6%30.8%21.5%7.7%
Thinking Machines Inkling22.0%30.0%22.0%29.0%23.0%
MiniMax M326.9%26.0%24.0%27.9%25.0%

Unauthorized cleanup

The agent deletes the user’s files or earlier work without permission.

Example: Deletes the user’s original images after editing copies of them.

Unrequested output

The agent produces something the user did not ask for, or keeps working after being told to stop.

Example: Generates an image when the user asked only for the prompt text.

Premature execution

The agent starts making changes before the user has approved them.

Example: Edits a file when asked to assess a proposed change.

Scope expansion

The agent changes parts of the work the user did not ask it to touch.

Example: Rewrites the body when asked only to change the title.

Protected-content rewriting

The agent changes content the user explicitly said to keep.

Example: Alters facts during a grammar-only edit.