# Vending-Bench 2 ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Bank%20Balance-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-1%20simulated%20year-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-2025-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2502.15840) [![Site](https://img.shields.io/badge/Site-andonlabs.com-1f6feb)](https://andonlabs.com/evals/vending-bench-2) > **Quick answer:** Vending-Bench 2 is Andon Labs' evolution of the original Vending-Bench, testing whether an LLM-based agent can coherently manage a simulated vending machine business — ordering inventory, setting prices, and paying daily fees — across a full simulated year, scored on the agent's final bank account balance. ## At a Glance **What it tests:** Long-horizon coherence — whether an agent can sustain consistent, non-derailing decision-making across thousands of turns and tens of millions of output tokens in a single continuous run. **Why it matters:** Short-task benchmarks can't reveal whether an agent's reasoning degrades, forgets earlier state, or spirals into unproductive loops over very long horizons; Vending-Bench 2 directly measures this by tying success to a single concrete, compounding financial outcome. **Known limitations:** Results have high run-to-run variance (reflected in the reported ± ranges), and because it's a single continuous simulated business rather than a set of independent tasks, results are less directly comparable to typical multi-task accuracy benchmarks. ## What Vending-Bench 2 Measures Building on the original Vending-Bench (Backlund & Petersson 2025), Vending-Bench 2 has agents operate a simulated vending machine business starting from a $500 balance, managing inventory ordering, pricing, and a daily $2 operating fee over a full simulated year. Runs typically span roughly 3,000–6,000 messages and 60–100M output tokens, and an agent's business is terminated if it fails to pay its operating fee for 10 consecutive days. The benchmark reports the average final bank balance across 5 runs per model, giving a single concrete financial outcome that reflects sustained planning, arithmetic, and recovery from mistakes over a very long horizon — rather than a percentage-correct score on isolated tasks. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent / long-horizon coherence | | Metric | Average final bank account balance ($) across 5 runs | | Number of tasks | 1 continuous simulated year-long business run per model | | Saturation | Low — final balances vary widely and no model reliably maximizes profit | | Created by | Andon Labs | | Source paper | [Backlund & Petersson 2025](https://arxiv.org/abs/2502.15840) (original Vending-Bench; v2 documented at [andonlabs.com](https://andonlabs.com/evals/vending-bench-2)) | | Official results | [andonlabs.com/evals/vending-bench-2](https://andonlabs.com/evals/vending-bench-2) | ## How Vending-Bench 2 Is Scored An agent's score is its average final bank account balance (in dollars) across 5 independent full-year simulation runs; higher balances indicate more successful, sustained business management, while runs that end in bankruptcy or termination cap the agent's earnings for that run. ## State-of-the-Art Results > Per Andon Labs' official leaderboard, the top scores are Claude Opus 5 ($11,181.87 ± $2,094), Claude Opus 4.7 ($10,936.76 ± $1,181), GPT-5.6 Sol ($9,619.37 ± $1,338), Grok 4.6 ($9,047.03 ± $1,604), and GLM-5.2 ($8,313.78 ± $1,084), with Claude Opus 4.6 further down the table at $8,017.59 ± $1,367. See the [official leaderboard](https://andonlabs.com/evals/vending-bench-2) for the full, continuously-updated ranking. ## Vending-Bench 2 on Benchgen > No Benchgen results yet — [be the first to run Vending-Bench 2](/benchmarks/andon-labs/vending-bench-2). ## Vending-Bench 2 vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **Vending-Bench 2** | Long-horizon agent coherence via a simulated business | 1 (year-long run) | low | | [WideSearch](/benchmarks/ryan-wong-jiawei-wang-junjie-zhao-li-chen-yan-gao-long-zhang-xuan-zhou-zuo-wang-kai-xiang-et-al-bytedance-seed/widesearch) | Broad, verifiable web information-gathering | 200 | low | | [OpenRCA](/benchmarks/junjielong-xu-qinan-zhang-zhiqing-zhong-shilin-he-chaoyun-zhang-qingwei-lin-dan-pei-pinjia-he-dongmei-zhang-qi-zhang-microsoft/openrca) | Root-cause analysis from enterprise telemetry | 335 | low | Vending-Bench 2 is distinguished from most agent benchmarks by scoring a single continuous, compounding simulation rather than many independent tasks, directly stress-testing whether an agent's coherence degrades over very long horizons. ## Run Vending-Bench 2 on Your Model Benchgen lets teams evaluate their own model's long-horizon coherence on Vending-Bench 2's simulated year-long vending business, surfacing where planning, memory, and recovery from mistakes break down over extended runs. ## Frequently Asked Questions

What is Vending-Bench 2?

Vending-Bench 2 is Andon Labs' long-horizon agent benchmark where models manage a simulated vending machine business for a full year, scored on their final bank account balance.

What does a good Vending-Bench 2 score look like?

On the official leaderboard, top models like Claude Opus 5 average over $11,000 in final balance across 5 runs, while weaker or less coherent agents can end with far lower balances or go bankrupt entirely.

Who created Vending-Bench 2?

Vending-Bench 2 was created by Andon Labs, building on the original Vending-Bench introduced by Axel Backlund and Lukas Petersson in 2025.

Rank	Model	Score
1	claude-opus-5	11181.87
2	claude-opus-4-7	10936.76
3	gpt-5-6-sol	9619.37
4	grok-4-6	9047.03
5	glm-5-2	8313.78
6	glm-5-3	8163.61
7	claude-opus-4-6	8017.59
8	gpt-5-5	7523.84
9	gpt-5-6-terra	7343.21
10	claude-sonnet-4-6	7204.14

Vending-Bench 2 — Results

Vending-Bench 2 — Leaderboard

Vending-Bench 2 Leaderboard