DeepSWE — Results
| Rank | Model | Score |
|---|---|---|
| 1 | kimi-k3 | 67.5 |
| 2 | grok-4-6 | 65.9 |
| 3 | muse-spark-1-2 | 59.3 |
| 4 | qwen3-8-max | 56.6 |
DeepSWE
1 phaseActive
Real-world software engineering agent benchmark curated by DataCurve, testing end-to-end coding-agent workflows. Metric: % task success.
Overview
DeepSWE
Quick answer: DeepSWE is a real-world software engineering agent benchmark curated by DataCurve, testing end-to-end coding agent workflows against verified ground-truth solutions. It's positioned alongside benchmarks like SWE-Bench as a harder, more diverse test of agentic coding capability. Kimi K3 scores 67.5% as of July 2026.
At a Glance
What it tests: An AI coding agent's ability to resolve real-world software engineering tasks end-to-end — understanding a codebase, implementing a fix or feature, and verifying correctness.
Why it matters: As SWE-Bench-style benchmarks approach saturation for frontier models, DeepSWE provides an independently curated task pool from DataCurve to cross-validate agentic coding performance.
Known limitations: As a newer, vendor-curated benchmark, public documentation of methodology and task composition is limited compared to established academic benchmarks like SWE-Bench.
What DeepSWE Measures
DeepSWE evaluates coding agents on real-world software engineering tasks curated by DataCurve, a data-curation company focused on agentic and coding evaluation datasets. Tasks are designed to mirror the type of work a software engineer performs day-to-day — bug fixes, feature implementation, and refactors — verified against reference solutions or test suites.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Coding agent |
| Metric | % task success |
| Saturation | Low |
| Created by | DataCurve |
| Dataset | datacurve/deep-swe on HuggingFace |
| Project | deepswe.datacurve.ai |
How DeepSWE Is Scored
Agents attempt each software engineering task and are scored on % of tasks resolved correctly, typically verified via automated test execution against the target repository.
State-of-the-Art Results
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Kimi K3 | 67.5% | Kimi K3 technical report | 2026-07 |
Score sourced from Moonshot AI's Kimi K3 announcement, July 2026.
DeepSWE on Benchgen
No Benchgen results yet — be the first to run DeepSWE.
DeepSWE vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| DeepSWE | Real-world software engineering agent tasks | Low |
| SWE-Bench Pro | Professional-grade SWE agent tasks | Low |
| SWE-Bench Verified | Verified real-world GitHub issue resolution | Medium |
| SWE-Marathon | Long-horizon software engineering | Low |
Run DeepSWE on Your Model
Benchgen lets you run DeepSWE against your own coding agent, tracking task success rate over time to catch regressions from model or harness updates.