ARC-AGI v2

ARC-AGI v2

Quick answer: ARC-AGI v2 is the second-generation Abstraction and Reasoning Corpus benchmark, introduced by François Chollet and collaborators in May 2025 to address the near-saturation of the original ARC-AGI. It uses the same visual grid transformation format — models must infer hidden rules from input/output examples and apply them to a test grid — but with significantly harder task designs that remain challenging even for frontier models. GPT-5.5 leads at 85.0%.

At a Glance

What it tests: Abstract visual reasoning: given a small number of input→output grid examples, infer the underlying transformation rule and apply it to a novel test grid. Requires spatial reasoning, pattern recognition, and compositional generalisation.

Why it matters: ARC-AGI v2 was designed specifically because the original ARC-AGI approached saturation for frontier models. V2 maintains the same evaluation philosophy — tests should be easy for humans but hard for current AI — while being substantially harder, preserving its value as a signal for true generalisation.

Known limitations: Scored on a 0–1 scale; tasks are not representative of all reasoning types (domain is visual/spatial). The benchmark is still early and the leaderboard is small relative to more established benchmarks.

What ARC-AGI v2 Measures

ARC-AGI v2 inherits the core ARC format: each task provides 2–5 input/output grid pairs demonstrating a hidden transformation. The model must identify the rule and produce the correct output for a held-out test input. Grids use up to 10 colours in cells ranging from 1×1 to 30×30.

V2 introduces harder task designs with more complex compositional rules, multi-step transformations, and higher sensitivity to spatial configuration. Where frontier models could approach 95%+ on the original ARC-AGI using test-time compute scaling, V2 creates meaningful differentiation — with a 50-point gap between the leader (85%) and models like Gemini 2.5 Pro (4.9%).

The benchmark is multimodal: models receive grids as visual inputs, making it relevant for evaluating vision-language models' abstract reasoning capabilities alongside text-only reasoning systems.

Benchmark Specifications

How ARC-AGI v2 Is Scored

Each task is binary: solved (1) or not solved (0). The score is the fraction of tasks solved. Models are typically allowed multiple attempts per task, and a task is counted correct if any attempt produces the right output. The 0–1 normalised scale means a score of 0.850 = 85% of tasks solved.

A score above 0.7 is considered frontier-tier. Human performance on ARC-AGI tasks approaches 1.0, making it the target ceiling.

ARC-AGI v2 on Benchgen

No Benchgen results yet — be the first to run ARC-AGI v2.

ARC-AGI v2 vs Other Benchmarks

ARC-AGI v2 is the go-to benchmark when you want to evaluate generalisation and fluid reasoning without saturation concerns that affect the original ARC-AGI.

Run ARC-AGI v2 on Your Model

Use Benchgen to run ARC-AGI v2 against your model continuously — tracking how reasoning capability evolves across fine-tuning runs, prompt changes, or model versions. Unlike one-time leaderboard submissions, Benchgen gives you a regression-tracked history of abstract reasoning performance.

Related Models & Benchmarks

• Top performers: GPT-5.5 · Gemini 3.1 Pro · GPT-5.4
• Related benchmarks: ARC-AGI · ARC-AGI-3 · GPQA Diamond

Frequently Asked Questions

Benchmark definition paraphrased from Chollet et al. 2025. State-of-the-art scores sourced from llm-stats.com and attributed inline. Last updated 2026-07-23.