Vending-Bench 2 — Results
| Rank | Model | Score |
|---|---|---|
| 1 | claude-opus-5 | 11181.87 |
| 2 | claude-opus-4-7 | 10936.76 |
| 3 | gpt-5-6-sol | 9619.37 |
| 4 | grok-4-6 | 9047.03 |
| 5 | glm-5-2 | 8313.78 |
| 6 | glm-5-3 | 8163.61 |
| 7 | claude-opus-4-6 | 8017.59 |
| 8 | gpt-5-5 | 7523.84 |
| 9 | gpt-5-6-terra | 7343.21 |
| 10 | claude-sonnet-4-6 | 7204.14 |
Vending-Bench 2
1 phaseActive
Andon Labs' long-horizon agent benchmark where models operate a simulated vending machine business for a full year, scored on final bank balance.
Overview
Vending-Bench 2
Quick answer: Vending-Bench 2 is Andon Labs' evolution of the original Vending-Bench, testing whether an LLM-based agent can coherently manage a simulated vending machine business — ordering inventory, setting prices, and paying daily fees — across a full simulated year, scored on the agent's final bank account balance.
At a Glance
What it tests: Long-horizon coherence — whether an agent can sustain consistent, non-derailing decision-making across thousands of turns and tens of millions of output tokens in a single continuous run.
Why it matters: Short-task benchmarks can't reveal whether an agent's reasoning degrades, forgets earlier state, or spirals into unproductive loops over very long horizons; Vending-Bench 2 directly measures this by tying success to a single concrete, compounding financial outcome.
Known limitations: Results have high run-to-run variance (reflected in the reported ± ranges), and because it's a single continuous simulated business rather than a set of independent tasks, results are less directly comparable to typical multi-task accuracy benchmarks.
What Vending-Bench 2 Measures
Building on the original Vending-Bench (Backlund & Petersson 2025), Vending-Bench 2 has agents operate a simulated vending machine business starting from a $500 balance, managing inventory ordering, pricing, and a daily $2 operating fee over a full simulated year. Runs typically span roughly 3,000–6,000 messages and 60–100M output tokens, and an agent's business is terminated if it fails to pay its operating fee for 10 consecutive days.
The benchmark reports the average final bank balance across 5 runs per model, giving a single concrete financial outcome that reflects sustained planning, arithmetic, and recovery from mistakes over a very long horizon — rather than a percentage-correct score on isolated tasks.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Agent / long-horizon coherence |
| Metric | Average final bank account balance ($) across 5 runs |
| Number of tasks | 1 continuous simulated year-long business run per model |
| Saturation | Low — final balances vary widely and no model reliably maximizes profit |
| Created by | Andon Labs |
| Source paper | Backlund & Petersson 2025 (original Vending-Bench; v2 documented at andonlabs.com) |
| Official results | andonlabs.com/evals/vending-bench-2 |
How Vending-Bench 2 Is Scored
An agent's score is its average final bank account balance (in dollars) across 5 independent full-year simulation runs; higher balances indicate more successful, sustained business management, while runs that end in bankruptcy or termination cap the agent's earnings for that run.
State-of-the-Art Results
Per Andon Labs' official leaderboard, the top scores are Claude Opus 5 ($11,181.87 ± $2,094), Claude Opus 4.7 ($10,936.76 ± $1,181), GPT-5.6 Sol ($9,619.37 ± $1,338), Grok 4.6 ($9,047.03 ± $1,604), and GLM-5.2 ($8,313.78 ± $1,084), with Claude Opus 4.6 further down the table at $8,017.59 ± $1,367. See the official leaderboard for the full, continuously-updated ranking.
Vending-Bench 2 on Benchgen
No Benchgen results yet — be the first to run Vending-Bench 2.
Vending-Bench 2 vs Other Benchmarks
| Benchmark | What it tests | Tasks | Saturation |
|---|---|---|---|
| Vending-Bench 2 | Long-horizon agent coherence via a simulated business | 1 (year-long run) | low |
| WideSearch | Broad, verifiable web information-gathering | 200 | low |
| OpenRCA | Root-cause analysis from enterprise telemetry | 335 | low |
Vending-Bench 2 is distinguished from most agent benchmarks by scoring a single continuous, compounding simulation rather than many independent tasks, directly stress-testing whether an agent's coherence degrades over very long horizons.
Run Vending-Bench 2 on Your Model
Benchgen lets teams evaluate their own model's long-horizon coherence on Vending-Bench 2's simulated year-long vending business, surfacing where planning, memory, and recovery from mistakes break down over extended runs.