# StaminaBench ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Turns%20Survived-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) > **Quick answer:** StaminaBench is an Amazon Science benchmark that measures coding-agent "stamina" — how many consecutive change-request turns (up to 100) an agent can sustain before it fails a request — as a black-box test of long-horizon reliability rather than single-shot task accuracy. It finds that access to test feedback and retry capability improves the number of turns survived by up to 12×, and that even the strongest models show a 6× gap between their best and worst harness. ## At a Glance **What it tests:** How many consecutive, iterative change-request turns a coding agent can handle in a single session before it breaks down — a proxy for long-horizon reliability rather than one-shot task accuracy. **Why it matters:** Real coding-agent usage is rarely single-shot; developers issue a stream of follow-up change requests in one session. StaminaBench directly measures the failure mode that matters in that setting — gradual degradation and eventual breakdown — rather than pass/fail on isolated tasks. **Known limitations:** As a black-box benchmark, it doesn't diagnose *why* an agent fails at turn N — only that it does. Detailed per-model, per-harness turn counts are reported in the source material but not fully reproduced here. ## What StaminaBench Measures StaminaBench places a coding agent into a long, iterative session and issues up to 100 sequential change requests, each building on the previous state of the code. The benchmark tracks the number of consecutive turns the agent can complete successfully before it fails a request — whether by introducing a regression, losing track of prior context, or simply refusing/erroring out. This "stamina" framing directly targets a failure mode that static, single-shot benchmarks can't see: agents that perform well on a first request often degrade steadily as session length grows, due to context management issues, compounding errors, or harness-level state drift. The benchmark's headline findings are both about harness design rather than raw model capability: giving the agent access to test feedback and retry capability (i.e., letting it see whether its own change broke something and try again) improves the number of turns survived by up to 12×. Separately, even the strongest models show as much as a 6× gap in turns survived between their best-performing harness and their worst — reinforcing that endurance under iterative use is substantially a harness property, not purely a model property. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent (coding, long-horizon endurance) | | Metric | Consecutive turns survived (count, capped at 100) | | Evaluation style | Black-box, iterative change-request sequence | | Saturation | Low | | Created by | Amazon Science | | Source | Amazon Science publication, 2026 | | GitHub | Not public | | Dataset | Not public | ## How StaminaBench Is Scored Each run starts a coding-agent session and issues change requests one at a time, in sequence, up to a maximum of 100 turns. The agent "survives" a turn if it completes the requested change without breaking existing functionality or losing coherent state; the run ends the moment the agent fails a turn (or reaches the 100-turn cap). The reported score is simply the number of turns survived — higher is better, and the 100-turn cap represents a ceiling rather than a typical result for most configurations. Because the benchmark isolates test feedback/retry access and harness choice as the two biggest levers on turns survived (12× and 6× respectively), it's best read as a harness-design diagnostic: teams building long-session coding agents can use it to check whether their retry/verification loop is actually extending session stamina. ## StaminaBench on Benchgen > No Benchgen results yet — [be the first to run StaminaBench](/benchmarks/staminabench). ## StaminaBench vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **StaminaBench** | Consecutive change-request turns survived (endurance) | Low | | [SWE-Marathon](/benchmarks/swe-marathon) | Long-horizon software engineering task completion | Low | | [SWE-bench Pro](/benchmarks/swe-bench-pro) | Harder single-shot software engineering tasks | Low | | [Harness-Bench](/benchmarks/harness-bench) | Harness quality on a fixed academic task suite | Low | StaminaBench and SWE-Marathon both target long-horizon coding-agent behavior, but StaminaBench specifically isolates iterative, multi-turn endurance (how many follow-up requests an agent can survive) rather than completion of a single extended task. Use StaminaBench when evaluating whether your agent's retry and verification loop holds up across a realistic multi-turn development session. ## Run StaminaBench on Your Model Benchgen lets teams run their own coding agent through an iterative, multi-turn change-request sequence and track how many turns it survives across model or harness updates — catching stamina regressions (e.g. a retry-loop change that quietly shortens session length) before they reach production developers. ## Related Models & Benchmarks - Related benchmarks: [SWE-Marathon](/benchmarks/swe-marathon) · [SWE-bench Pro](/benchmarks/swe-bench-pro) · [Harness-Bench](/benchmarks/harness-bench) ## Frequently Asked Questions

What is StaminaBench?

StaminaBench is an Amazon Science benchmark that measures how many consecutive change-request turns (up to 100) a coding agent can survive in a single session before failing — a test of long-horizon reliability rather than single-shot accuracy.

What does a good StaminaBench score look like?

Scores are reported as the number of consecutive turns survived, capped at 100. Access to test feedback and retry capability improves turns survived by up to 12×, and harness choice alone produces up to a 6× gap even for the strongest models.

Who created StaminaBench?

StaminaBench was created by Amazon Science in 2026 as a black-box benchmark for coding-agent endurance under iterative use.

Why does harness choice matter so much on StaminaBench?

Endurance across many consecutive turns depends heavily on how the harness manages context, verifies changes, and handles retries — StaminaBench found up to a 6× difference in turns survived between the best and worst harness for the same model.

Is StaminaBench saturated?

No. Most tested configurations fail well before the 100-turn cap, and the wide spread between harnesses (6×) and feedback conditions (12×) shows the benchmark still meaningfully differentiates agent endurance.

StaminaBench — Evaluations