Nvidia pushed Claude from 30% to 100%: the real Agent advantage may not be the model
Nvidia reports that Claude Opus 5 rose from about 30% to 100% on the ARC-AGI-3 interactive-reasoning test after adding the AVO Harness's memory, tool loop and supervising Agent rather than changing the model. It is one benchmark with a vendor-built configuration, not a universal production result, but it shows why memory, retries and supervision may matter as much as model rankings.
The short version
Nvidia did not release a new model, yet it pushed the same Agent from 30% to 100%.
In research published on August 21, Nvidia placed Claude Opus 5 inside a more elaborate Agent Harness: better memory, a tool loop, and a supervising agent that pulls the main Agent out of dead ends. On the ARC-AGI-3 interactive-reasoning test, the bare model scored about 30%; with the Harness, it reached 100%.
This does not mean Claude has beaten every model, and it does not mean every task will improve by 70 points. The real finding is that a large part of long-horizon performance comes from the system around the model.
What happened
Nvidia defines a Harness as the tools, memory, runtime and rules wrapped around a model. It determines how an Agent reads context, calls tools, records attempts, detects failure and replans.
The Agentic Variation Operators (AVO) system used in this study also adds a supervising Agent. The main Agent solves the task; the supervisor acts like a small project lead, nudging it when it repeats a path, drifts from the goal or gets stuck.
ARC-AGI-3 is a set of interactive 2D games with no instructions. An Agent must observe the environment, infer the rules, act repeatedly and revise its strategy from feedback. That makes it useful for exposing memory breaks and compounding errors in long tasks.
The number that is easiest to misread
TechCrunch reported Nvidia's result as roughly 30% for Claude Opus 5 without the Harness and 100% with AVO. The gap is striking, but it comes from one benchmark and one Nvidia-built configuration.
It does not establish that:
- AVO will reach 100% on real software-engineering work;
- Claude Opus 5 is now the best general model;
- ordinary users can reproduce the result with a better prompt.
The safer fact is this: the same model can change dramatically when its execution system changes. OpenAI also reported earlier this year that changing context retention and compression raised GPT-5.6 Sol from 13.3% to 38.3% on ARC-AGI-3. The experiments are not identical, but they point in the same direction.
The reversal: model pricing may not be model pricing
Agent cost is not determined only by the price per million tokens.
If a Harness rereads files, stores longer memories, starts a supervisor and retries after failure, the same model can consume very different amounts of tokens in different systems. Databricks has also argued that a poor Harness choice can double the cost of the same model.
So the useful comparison is at least:
- tokens consumed per task;
- attempts needed to succeed;
- rework and human-review time;
- final cost per mergeable result.
What developers can use
If you build coding agents, browser agents or enterprise automation, four changes may matter before switching models:
- Store critical state in recoverable memory instead of relying only on the context window.
- Make tools return structured results so the Agent does not have to guess why something failed.
- Add an independent checker for long tasks and interrupt repeated actions.
- Record cost, failure reasons and whether the final result needed human rework.
That does not mean every product needs multiple Agents. A supervisor adds latency, complexity and cost, and may not be worthwhile for short answers or one-shot completions.
Our judgment
The important value of Nvidia's research is not that one model reached a perfect score. It moves attention from model leaderboards to complete Agent systems.
That also helps explain why the same Grok, Claude or GPT can feel very different inside different coding tools. The model is the brain; the Harness determines whether it has memory, hands and brakes.
Still, AVO is not yet a general production recipe. The result comes from a vendor-built Harness, the test scale is limited, and security, permissions and maintenance costs in real software projects have not been validated to the same degree.
What to watch next
- Whether Nvidia releases enough code for outside teams to reproduce the result.
- Whether AVO remains effective on coding, browser and enterprise-workflow benchmarks.
- Whether the accuracy gain from a supervisor justifies extra tokens and latency.
- Whether open Harnesses build ecosystem moats faster than closed models do.
For ordinary users, inspect memory, retries, approvals and verification in your current Agent before switching models again.