# DeepSWE ![Category](https://img.shields.io/badge/Category-Coding%20Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Success-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/datacurve/deep-swe) > **Quick answer:** DeepSWE is a real-world software engineering agent benchmark curated by DataCurve, testing end-to-end coding agent workflows against verified ground-truth solutions. It's positioned alongside benchmarks like SWE-Bench as a harder, more diverse test of agentic coding capability. Kimi K3 scores 67.5% as of July 2026. ## At a Glance **What it tests:** An AI coding agent's ability to resolve real-world software engineering tasks end-to-end — understanding a codebase, implementing a fix or feature, and verifying correctness. **Why it matters:** As SWE-Bench-style benchmarks approach saturation for frontier models, DeepSWE provides an independently curated task pool from DataCurve to cross-validate agentic coding performance. **Known limitations:** As a newer, vendor-curated benchmark, public documentation of methodology and task composition is limited compared to established academic benchmarks like SWE-Bench. ## What DeepSWE Measures DeepSWE evaluates coding agents on real-world software engineering tasks curated by DataCurve, a data-curation company focused on agentic and coding evaluation datasets. Tasks are designed to mirror the type of work a software engineer performs day-to-day — bug fixes, feature implementation, and refactors — verified against reference solutions or test suites. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Coding agent | | Metric | % task success | | Saturation | Low | | Created by | DataCurve | | Dataset | [datacurve/deep-swe on HuggingFace](https://huggingface.co/datasets/datacurve/deep-swe) | | Project | [deepswe.datacurve.ai](https://deepswe.datacurve.ai) | ## How DeepSWE Is Scored Agents attempt each software engineering task and are scored on % of tasks resolved correctly, typically verified via automated test execution against the target repository. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Kimi K3](/models/moonshot/kimi-k3) | 67.5% | [Kimi K3 technical report](https://www.kimi.com/blog/kimi-k3) | 2026-07 | *Score sourced from Moonshot AI's Kimi K3 announcement, July 2026.* ## DeepSWE on Benchgen > No Benchgen results yet — [be the first to run DeepSWE](/benchmarks/deepswe). ## DeepSWE vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **DeepSWE** | Real-world software engineering agent tasks | Low | | [SWE-Bench Pro](/benchmarks/swe-bench-pro) | Professional-grade SWE agent tasks | Low | | [SWE-Bench Verified](/benchmarks/swe-bench-verified) | Verified real-world GitHub issue resolution | Medium | | [SWE-Marathon](/benchmarks/swe-marathon) | Long-horizon software engineering | Low | ## Run DeepSWE on Your Model Benchgen lets you run DeepSWE against your own coding agent, tracking task success rate over time to catch regressions from model or harness updates. ## Frequently Asked Questions

What is DeepSWE?

DeepSWE is a real-world software engineering agent benchmark curated by DataCurve, testing coding agents on end-to-end development tasks verified against reference solutions.

What does a good score look like on DeepSWE?

Kimi K3 reports 67.5% as of July 2026; scores in this range are considered strong for current frontier coding agents on real-world software engineering workflows.

Who created DeepSWE?

DeepSWE is curated by DataCurve, a company specializing in data curation for agentic and coding model evaluation.

Rank	Model	Score
1	deepseek-v4-1-flash	74.2
2	deepseek-v4-1-flash	74.2
3	gpt-6-astra	74.1
4	grok-4-7	71
5	gpt-6-sol	68.8
6	kimi-k3	67.5
7	glm-5-3	66.9
8	gpt-6-luna	66.6
9	grok-4-6	65.9
10	nex-n2-5-max	65.6
11	hy4-preview	64.3
12	glm-5-3-flash	63.4
13	deepseek-v4-pro	62.7
14	muse-spark-1-2	59.3
15	deepseek-v4-flash-vision-exp	59.3
16	qwen3-8-flash-next	58.7
17	qwen3-8-max	56.6
18	ornith-1-5-397b	56
19	nex-n2-5-pro	55.8
20	qwen3-8-27b	42.2
21	ornith-1-5-35b-a3b	22

DeepSWE — Results

DeepSWE — Evaluations