# Artificial Analysis Coding Agent Index ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Composite%20Index%20%2B%20Cost-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) [![Leaderboard](https://img.shields.io/badge/Leaderboard-artificialanalysis.ai-0969da)](https://artificialanalysis.ai/agents/coding-agents) > **Quick answer:** The Artificial Analysis Coding Agent Index is a public, ongoing leaderboard that composites three benchmarks — DeepSWE, Terminal-Bench v2, and SWE-Atlas-QnA — to rank coding-agent *harness–model combinations* rather than models alone. Its most-cited finding: the same model (e.g. Claude Opus 4.7) run through different harnesses (Claude Code vs. Cursor CLI vs. OpenCode) shows a 32× swing in cost per task ($0.07–$2.26) at near-identical code quality, making harness choice one of the largest cost levers in production coding-agent deployments. ## At a Glance **What it tests:** Coding-agent quality *and* operating cost, reported per harness–model combination, holding the model constant to isolate what the harness itself changes. **Why it matters:** It's one of the most visible public leaderboards explicitly built around the "same model, different harness" comparison — turning harness choice into a measurable, board-worthy line item (cost per task, time per task) rather than an invisible implementation detail. **Known limitations:** As a composite index, its score aggregates three underlying benchmarks with different task distributions and difficulty profiles; teams should check the individual component scores (DeepSWE, Terminal-Bench v2, SWE-Atlas-QnA) if they care about a specific coding-agent capability rather than the blended index. ## What the Coding Agent Index Measures The Coding Agent Index blends results from three existing coding-agent benchmarks — DeepSWE, Terminal-Bench v2, and SWE-Atlas-QnA — into a single composite score, then reports that score per harness–model combination instead of per model alone. This design choice reflects a growing consensus (also seen in Harness-Bench and EnterpriseClawBench) that a model's real-world coding-agent quality is inseparable from the harness wrapped around it: tool routing, retry logic, context management, and prompt scaffolding all materially change outcomes even when the underlying model is identical. Alongside the composite quality score, Artificial Analysis reports cost per task and time per task for each harness–model pair — because in production deployments, a harness that's marginally better on quality but dramatically more expensive or slower is often the wrong choice. The index's most widely cited data point is a 32× cost swing ($0.07 to $2.26 per task) observed across different harnesses running the *same* model, with near-identical resulting code quality — meaning the harness alone, not the model, was responsible for most of the cost difference. Because it's a living index rather than a static paper, rankings shift as harnesses (Claude Code, Cursor CLI, OpenCode, and others) release updates and as new frontier models are added to the comparison grid. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent (coding, harness comparison) | | Metric | Composite index score + cost ($) and time per task | | Component benchmarks | DeepSWE, Terminal-Bench v2, SWE-Atlas-QnA | | Leaderboard unit | Harness–model combination | | Saturation | Low | | Created by | Artificial Analysis | | Leaderboard | [artificialanalysis.ai/agents/coding-agents](https://artificialanalysis.ai/agents/coding-agents) | | Paper | Not applicable — ongoing public leaderboard, not a static paper | | Dataset | Not public (aggregates existing benchmark datasets) | ## How the Coding Agent Index Is Scored Each harness–model combination is run against the three component benchmarks, and the resulting scores are blended into a single composite index value. Cost per task and time per task are tracked separately and reported alongside the quality score rather than folded into it, so teams can see the full quality/cost/speed trade-off for each harness–model pair rather than a single number that obscures cost differences. The headline 32× cost gap ($0.07–$2.26 per task) was observed between harnesses running the identical model (Claude Opus 4.7) with comparable output quality — illustrating that, on this index, harness choice can be a larger lever on operating cost than model choice. ## State-of-the-Art Results | Metric | Observation | Source | Date | |---|---|---|---| | Cost range across harnesses (same model, Claude Opus 4.7) | $0.07 – $2.26 per task (32×) | [Coding Agent Index coverage](https://medium.com/@wasowski.jarek/coding-agent-index-2026-benchmarking-full-agent-stacks-model-harness-4183305e4b90) | 2026 | | Harnesses compared | Claude Code, Cursor CLI, OpenCode | [Artificial Analysis](https://artificialanalysis.ai/agents/coding-agents) | 2026 | *The full ranked leaderboard (harness–model composite scores) updates continuously on Artificial Analysis and is not reproduced in static form here — see the live leaderboard for current standings.* ## Coding Agent Index on Benchgen > No Benchgen results yet — [be the first to run a harness-cost comparison](/benchmarks/aa-coding-agent-index). ## Coding Agent Index vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **AA Coding Agent Index** | Coding-agent quality + cost, per harness–model pair | Low | | [Harness-Bench](/benchmarks/harness-bench) | Harness quality on a fixed academic task suite | Low | | [EnterpriseClawBench](/benchmarks/enterprise-claw-bench) | Enterprise agent workflows, harness–model pairs | Low | | [SWE-bench Pro](/benchmarks/swe-bench-pro) | Harder software engineering tasks, model-only | Low | The Coding Agent Index is the most visible *commercial* leaderboard doing harness-vs-model breakdowns, while Harness-Bench and EnterpriseClawBench are academic benchmarks with similar goals. Use the Coding Agent Index when cost-per-task and time-per-task matter as much as raw quality; use Harness-Bench when you need a controlled, oracle-checkable task suite. ## Run a Harness Cost Comparison on Your Model Benchgen lets teams benchmark their own harness–model combinations for quality, cost, and time per task — so you can make the same trade-off decisions the Coding Agent Index highlights, but against your own production configuration instead of a public reference model. ## Related Models & Benchmarks - Related models: [Claude Opus 4.7](/models/anthropic/claude-opus-4-7) · [GPT-5.5](/models/openai/gpt-5-5) - Related benchmarks: [Harness-Bench](/benchmarks/harness-bench) · [EnterpriseClawBench](/benchmarks/enterprise-claw-bench) · [SWE-bench Pro](/benchmarks/swe-bench-pro) ## Frequently Asked Questions

What is the Artificial Analysis Coding Agent Index?

It's a public, continuously updated leaderboard from Artificial Analysis that composites three coding-agent benchmarks (DeepSWE, Terminal-Bench v2, SWE-Atlas-QnA) and ranks results by harness–model combination rather than by model alone.

What is the 32× cost finding on the Coding Agent Index?

Running the same model (Claude Opus 4.7) through different harnesses (Claude Code, Cursor CLI, OpenCode) produced a cost range of $0.07 to $2.26 per task — a 32× difference — despite near-identical resulting code quality.

Who created the Coding Agent Index?

The Coding Agent Index is maintained by Artificial Analysis as an ongoing public leaderboard, not a static academic paper.

Is the Coding Agent Index saturated?

No. Large swings in both quality and cost persist across harness–model combinations, and rankings shift as harnesses and models are updated, so the index continues to meaningfully differentiate configurations.

How does the Coding Agent Index differ from Harness-Bench?

The Coding Agent Index is a commercial, continuously updated leaderboard focused on cost and quality trade-offs for popular production harnesses (Claude Code, Cursor CLI, OpenCode). Harness-Bench is an academic benchmark using a fixed, oracle-checkable 106-task suite specifically designed to isolate harness effects from model effects.

Rank	Model	Score
1	gpt-6-astra	67
2	mistral-large-4	48

Artificial Analysis Coding Agent Index — Results

Artificial Analysis Coding Agent Index — Evaluations