Agents' Last Exam

Agents' Last Exam

Quick answer: Agents' Last Exam (ALE) is a benchmark from UC Berkeley's RDI group (Sun et al., June 2026) that evaluates AI agents on 1,000+ long-horizon, economically valuable real-world tasks spanning 55 sub-fields across 13 industry clusters — built with 250+ industry experts using the O*NET/SOC 2018 occupational taxonomy. Unlike academic benchmarks, ALE is designed as a living benchmark specifically to close the gap between leaderboard performance and GDP-relevant impact.

At a Glance

What it tests: An AI agent's ability to complete sustained, multi-step professional workflows with verifiable outcomes across non-physical industries — from legal research to financial analysis, data science, and beyond.

Why it matters: Most agent benchmarks use synthetic or simplified tasks that don't map to actual work that generates economic value. ALE was co-designed with 250+ industry practitioners to ensure every task represents a genuine, high-value workflow. The hardest tier sees average pass rates below 1%, making it a strong differentiator between frontier models even as easier benchmarks saturate.

Known limitations: As a living benchmark, task difficulty and coverage evolve over time, making direct comparisons across evaluation dates unreliable without version pinning. The full hardest tier is extremely difficult, so aggregate scores primarily reflect performance on easier sub-tiers.

What Agents' Last Exam Measures

ALE organises tasks using the U.S. federal occupational taxonomy (O*NET/SOC 2018) as a structural backbone, covering non-physical industries across 13 clusters and 55 sub-fields. This grounds the benchmark in work that is already defined, scoped, and valued in the real economy — rather than researcher-invented synthetic tasks. Each task has a verifiable outcome, enabling automated evaluation without LLM-as-judge.

The benchmark is structured into difficulty tiers. The full-task ("hardest") tier requires complete end-to-end execution of a professional workflow, where current state-of-the-art models average below 1% pass rate. Easier tiers decompose tasks into sub-steps, producing more informative scores for current models. The leaderboard reports results across harness and backbone configurations, reflecting that agent benchmarks evaluate harness-model pairs, not models in isolation.

ALE is explicitly designed as a living benchmark: its task pool grows continuously as new industries and workflows are onboarded, and the benchmark is intended to remain unsaturated as AI capabilities advance.

Benchmark Specifications

How Agents' Last Exam Is Scored

Each task is scored pass/fail based on whether the agent's output satisfies the verifiable outcome criterion. The benchmark reports a pass rate (fraction of tasks passed) per tier and per industry cluster. Because ALE evaluates harness-model pairs, results depend heavily on the scaffolding used — the same backbone model can achieve substantially different scores with different agent frameworks.

State-of-the-Art Results

Scores sourced from llm-stats.com leaderboard. ALE evaluates harness-model pairs — scores vary significantly by scaffolding. See each source for methodology.

Agents' Last Exam on Benchgen

No Benchgen results yet — be the first to run Agents' Last Exam.

Agents' Last Exam vs Other Benchmarks

ALE's unique strength is its breadth across 13 real-economy industries and its grounding in O*NET occupational taxonomy — use it for cross-domain agent capability assessment. Use domain-specific benchmarks when you need detailed coverage of a single vertical.

Run Agents' Last Exam on Your Model

Benchgen lets you run Agents' Last Exam against your own model and harness configurations, version-tracking results across deployments so you can detect regressions and measure the real impact of fine-tuning on economically relevant task performance.

Frequently Asked Questions