# SWE Atlas – Refactoring ![Category](https://img.shields.io/badge/Category-Coding-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Accuracy%2FRubric-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-70-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2605.08366) [![GitHub](https://img.shields.io/badge/GitHub-scaleapi%2FSWE--Atlas-181717?logo=github&logoColor=white)](https://github.com/scaleapi/SWE-Atlas) > **Quick answer:** SWE Atlas – Refactoring is one of three sub-benchmarks in Scale AI's SWE Atlas suite. It tests whether a coding agent can perform correct, high-quality code refactors — 70 tasks scored with programmatic checks plus rubric-based assessment. ## At a Glance **What it tests:** Whether an AI coding agent can restructure existing code (improving readability, structure, or maintainability) without changing its observable behavior. **Why it matters:** Refactoring is one of the most common, highest-value real engineering tasks — and one of the easiest for an agent to get subtly wrong by silently changing behavior. **Known limitations:** Small task count (70); "refactoring quality" beyond behavior-preservation is judged by rubric, which is inherently more subjective than a binary pass/fail check. ## What SWE Atlas – Refactoring Measures This slice of Scale AI's SWE Atlas suite asks an agent to refactor specified code in a real repository — for example, extracting a function, simplifying control flow, or improving naming — while preserving exact behavior. Grading combines programmatic checks (do existing tests still pass, is behavior preserved) with expert rubric review of refactoring quality (did the change actually improve the code). A high score indicates an agent can make codebases measurably better without breaking anything — a core, everyday agentic-coding capability distinct from either comprehension (Codebase Q&A) or new-test authorship (Test Writing). ## Benchmark Specifications | Field | Value | |---|---| | Task category | Coding | | Metric | Accuracy / rubric score | | Number of tasks | 70 | | Saturation | Low | | Created by | Raghavendra et al. (Scale AI) | | Source paper | [Raghavendra et al. 2026](https://arxiv.org/abs/2605.08366) | | GitHub | [scaleapi/SWE-Atlas](https://github.com/scaleapi/SWE-Atlas) | ## How SWE Atlas – Refactoring Is Scored Scores combine automated behavior-preservation checks (existing tests must still pass) with expert rubric assessment of refactoring quality, reported as a percentage. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Hy4 Preview](/models/hungyuan/hy4-preview) | 53.3% | [Tencent Hunyuan model card](https://huggingface.co/tencent/Hy4-preview) | 2026-08 | *Scores sourced from published technical reports and model cards. Results depend on harness, prompt format, and effort settings — see each source for methodology.* ## SWE Atlas – Refactoring on Benchgen > No Benchgen results yet — [be the first to run SWE Atlas – Refactoring](/benchmarks/raghavendra-et-al-scale-ai/swe-atlas-refactoring). ## SWE Atlas – Refactoring vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **SWE Atlas – Refactoring** | Code refactoring quality | 70 | Low | | [SWE Atlas – Codebase Q&A](/benchmarks/raghavendra-et-al-scale-ai/swe-atlas-codebase-qa) | Repository comprehension | 124 | Low | | [SWE Atlas – Test Writing](/benchmarks/raghavendra-et-al-scale-ai/swe-atlas-test-writing) | Writing tests for existing code | 90 | Low | Use SWE Atlas – Refactoring to evaluate an agent's ability to safely improve existing code, distinct from its ability to resolve new issues from scratch. ## Run SWE Atlas – Refactoring on Your Model Benchgen lets teams run SWE Atlas – Refactoring against their own model versions, compare results across runs and harnesses, and catch regressions in refactoring quality over time. ## Related Models & Benchmarks - Top performers: [Hy4 Preview](/models/hungyuan/hy4-preview) - Related benchmarks: [SWE Atlas – Codebase Q&A](/benchmarks/raghavendra-et-al-scale-ai/swe-atlas-codebase-qa) · [SWE Atlas – Test Writing](/benchmarks/raghavendra-et-al-scale-ai/swe-atlas-test-writing) ## Frequently Asked Questions

What is SWE Atlas – Refactoring?

It's a sub-benchmark of Scale AI's SWE Atlas suite that tests whether a coding agent can perform correct, high-quality code refactors, across 70 tasks.

What does a good SWE Atlas – Refactoring score look like?

As of August 2026, frontier models score in the 35–55% range; scores above 50% represent strong, reliable refactoring capability.

Who created SWE Atlas?

SWE Atlas was created by Scale AI (Raghavendra et al.), published in 2026.

SWE Atlas – Refactoring — Results

SWE Atlas – Refactoring — Leaderboard

SWE Atlas – Refactoring Leaderboard