# SkillsBench ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Resolution%20Rate-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-87-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-Jun%202026-555) [![Paper](https://img.shields.io/badge/Paper-arXiv%202602.12670-b31b1b)](https://arxiv.org/abs/2602.12670) [![Dataset](https://img.shields.io/badge/Dataset-skillsbench.ai-ff9d00)](https://www.skillsbench.ai/blogs/skillsbench-1-1) > **Quick answer:** SkillsBench 1.1 is a June 2026 benchmark from the BenchFlow Team (advised by Dawn Song) that measures whether "Skills" — curated, reusable context/instruction packages attached to an agent harness — actually improve task resolution. Across 87 tasks in 8 domains and 18 model–harness configurations, curated Skills raise mean resolution rate from 33.9% to 50.5% (+16.6 points), while self-generated Skills (written by the agent itself) *reduce* performance by 8–11.5 points versus using no Skills at all. ## At a Glance **What it tests:** Whether adding Skills (curated instruction/context packages) to an agent harness improves task resolution, using a paired design that runs the same tasks with no Skills, curated Skills, and self-generated Skills. **Why it matters:** Skills and similar context-injection mechanisms (Claude Code Skills, MCP-based context packs) are a fast-growing harness-design pattern. SkillsBench is one of the first benchmarks to rigorously separate "Skills that help" from "Skills that hurt," with a concrete, counter-intuitive finding about self-generated Skills. **Known limitations:** Covers 4 harnesses (Claude Code, Codex, Gemini CLI, OpenHands) and 87 tasks — a meaningful but not exhaustive sample of harness/domain combinations. Results may not generalize to harnesses with fundamentally different context-management designs. ## What SkillsBench Measures SkillsBench 1.1 uses a paired evaluation design: every task in its 87-task, 8-domain suite is run three ways — with no Skills (baseline), with curated Skills (human- or expert-authored instruction packages), and with self-generated Skills (Skills the agent writes for itself mid-task). This produces 18 model–harness configurations across 4 harnesses (Claude Code, Codex, Gemini CLI, OpenHands), for a total of 9,396 scored trajectories. The paired design is what makes the benchmark's findings credible: because the same task is run under all three Skills conditions, differences in resolution rate can be attributed directly to the Skills condition rather than to task difficulty variance. The headline result — curated Skills lifting mean resolution from 33.9% to 50.5%, a 16.6-point gain — demonstrates that well-designed context packages are a genuine harness-level performance lever, not just a convenience feature. The more surprising finding is that self-generated Skills *hurt* performance by 8 to 11.5 points relative to using no Skills at all. This suggests that letting an agent author its own reusable context mid-task, without curation or review, introduces enough noise or incorrect generalization to offset any benefit — an important caution for harness designs that lean on agent self-improvement loops without a verification step. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent (Skills / context-package evaluation) | | Metric | Task resolution rate (%) | | Number of tasks | 87 across 8 domains | | Harnesses evaluated | Claude Code, Codex, Gemini CLI, OpenHands | | Configurations | 18 model–harness combinations | | Total trajectories | 9,396 | | Distribution | Prime Intellect Environments Hub, AgentBeats, Harbor | | Saturation | Low (best mean resolution: 50.5%) | | Created by | BenchFlow Team (advised by Dawn Song) | | Source paper | [SkillsBench 1.1 (arXiv 2602.12670)](https://arxiv.org/abs/2602.12670) | | Dataset | [skillsbench.ai](https://www.skillsbench.ai/blogs/skillsbench-1-1) | ## How SkillsBench Is Scored Each task is scored as resolved or unresolved (task resolution rate), and the benchmark reports the mean resolution rate separately for each of the three Skills conditions (no Skills, curated Skills, self-generated Skills), averaged across the 18 model–harness configurations. The gap between conditions — rather than the absolute resolution rate on any single configuration — is the benchmark's primary signal, since it isolates the causal effect of the Skills mechanism itself. ## State-of-the-Art Results Scores below are mean resolution rates averaged across all 18 model–harness configurations, broken out by Skills condition — the benchmark's core comparison. | Rank | Configuration | Mean resolution rate | Source | Date | |---|---|---|---|---| | 1 | Curated Skills | 50.5% | [SkillsBench 1.1](https://www.skillsbench.ai/blogs/skillsbench-1-1) | 2026-06 | | 2 | No Skills (baseline) | 33.9% | [SkillsBench 1.1](https://www.skillsbench.ai/blogs/skillsbench-1-1) | 2026-06 | | 3 | Self-generated Skills | ~22–26% (33.9% − 8 to 11.5 pts) | [SkillsBench 1.1](https://www.skillsbench.ai/blogs/skillsbench-1-1) | 2026-06 | *Scores sourced from the SkillsBench 1.1 release notes. Results depend on harness, domain mix, and Skill authoring quality — see the source for full per-harness and per-domain breakdowns.* ## SkillsBench on Benchgen > No Benchgen results yet — [be the first to run SkillsBench](/benchmarks/benchflow-team-advised-by-dawn-song/skillsbench). ## SkillsBench vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **SkillsBench** | Effect of Skills/context packages on harness performance | 87 | Low | | [Harness-Bench](/benchmarks/peking-university-qiyuan-tech/harness-bench) | Harness quality, holding model & task fixed | 106 | Low | | [MCP Atlas](/benchmarks/scale-ai/mcp-atlas) | MCP tool-use in agentic workflows | — | Low | | [EnterpriseClawBench](/benchmarks/zhong-et-al-frontisai/enterpriseclawbench-69) | Enterprise agent workflows (harness–model pairs) | 852 | Low | SkillsBench is the most targeted benchmark for evaluating a single harness-design pattern — Skills/context packages — in isolation, whereas Harness-Bench and EnterpriseClawBench evaluate the harness as a whole. Use SkillsBench when deciding whether to invest in curated Skills authoring versus letting agents self-generate context. ## Run SkillsBench on Your Model Benchgen lets teams test whether curated Skills, self-generated Skills, or no Skills produces the best resolution rate on their own harness and task distribution — turning SkillsBench's paired methodology into a repeatable check before shipping a Skills-based context strategy to production. ## Related Models & Benchmarks - Related benchmarks: [Harness-Bench](/benchmarks/peking-university-qiyuan-tech/harness-bench) · [MCP Atlas](/benchmarks/scale-ai/mcp-atlas) · [EnterpriseClawBench](/benchmarks/zhong-et-al-frontisai/enterpriseclawbench-69) ## Frequently Asked Questions

What is SkillsBench?

SkillsBench is a benchmark that measures whether Skills — curated, reusable instruction/context packages attached to an agent harness — improve task resolution, using a paired evaluation across 87 tasks, 8 domains, and 4 harnesses.

What does a good SkillsBench score look like?

Mean resolution rate with curated Skills is 50.5%, versus 33.9% with no Skills — a 16.6-point improvement. Self-generated Skills score lower than no Skills at all, around 22–26%.

Who created SkillsBench?

SkillsBench 1.1 was created by the BenchFlow Team, advised by Dawn Song, published in June 2026 (arXiv:2602.12670).

Why do self-generated Skills hurt performance on SkillsBench?

When agents write their own reusable Skills mid-task without curation or review, the resulting instructions introduce noise or incorrect generalizations, reducing mean resolution rate by 8–11.5 points compared to using no Skills at all.

Is SkillsBench saturated?

No. Even the best-performing condition (curated Skills) only reaches 50.5% mean resolution rate, leaving substantial headroom across all 18 evaluated configurations.

Rank	Model	Score
1	ling-3-1-flash-1019	68.7
2	ling-3-1-flash-1019	68.7
3	nanbeige4-3-33b	61.1
4	dots3-note-preview	52.8
5	muse-glimmer	44.3

SkillsBench — Results

SkillsBench — Evaluations