SkillsBench

SkillsBench

Quick answer: SkillsBench 1.1 is a June 2026 benchmark from the BenchFlow Team (advised by Dawn Song) that measures whether "Skills" — curated, reusable context/instruction packages attached to an agent harness — actually improve task resolution. Across 87 tasks in 8 domains and 18 model–harness configurations, curated Skills raise mean resolution rate from 33.9% to 50.5% (+16.6 points), while self-generated Skills (written by the agent itself) reduce performance by 8–11.5 points versus using no Skills at all.

At a Glance

What it tests: Whether adding Skills (curated instruction/context packages) to an agent harness improves task resolution, using a paired design that runs the same tasks with no Skills, curated Skills, and self-generated Skills.

Why it matters: Skills and similar context-injection mechanisms (Claude Code Skills, MCP-based context packs) are a fast-growing harness-design pattern. SkillsBench is one of the first benchmarks to rigorously separate "Skills that help" from "Skills that hurt," with a concrete, counter-intuitive finding about self-generated Skills.

Known limitations: Covers 4 harnesses (Claude Code, Codex, Gemini CLI, OpenHands) and 87 tasks — a meaningful but not exhaustive sample of harness/domain combinations. Results may not generalize to harnesses with fundamentally different context-management designs.

What SkillsBench Measures

SkillsBench 1.1 uses a paired evaluation design: every task in its 87-task, 8-domain suite is run three ways — with no Skills (baseline), with curated Skills (human- or expert-authored instruction packages), and with self-generated Skills (Skills the agent writes for itself mid-task). This produces 18 model–harness configurations across 4 harnesses (Claude Code, Codex, Gemini CLI, OpenHands), for a total of 9,396 scored trajectories.

The paired design is what makes the benchmark's findings credible: because the same task is run under all three Skills conditions, differences in resolution rate can be attributed directly to the Skills condition rather than to task difficulty variance. The headline result — curated Skills lifting mean resolution from 33.9% to 50.5%, a 16.6-point gain — demonstrates that well-designed context packages are a genuine harness-level performance lever, not just a convenience feature.

The more surprising finding is that self-generated Skills hurt performance by 8 to 11.5 points relative to using no Skills at all. This suggests that letting an agent author its own reusable context mid-task, without curation or review, introduces enough noise or incorrect generalization to offset any benefit — an important caution for harness designs that lean on agent self-improvement loops without a verification step.

Benchmark Specifications

How SkillsBench Is Scored

Each task is scored as resolved or unresolved (task resolution rate), and the benchmark reports the mean resolution rate separately for each of the three Skills conditions (no Skills, curated Skills, self-generated Skills), averaged across the 18 model–harness configurations. The gap between conditions — rather than the absolute resolution rate on any single configuration — is the benchmark's primary signal, since it isolates the causal effect of the Skills mechanism itself.

State-of-the-Art Results

Scores below are mean resolution rates averaged across all 18 model–harness configurations, broken out by Skills condition — the benchmark's core comparison.

Scores sourced from the SkillsBench 1.1 release notes. Results depend on harness, domain mix, and Skill authoring quality — see the source for full per-harness and per-domain breakdowns.

SkillsBench on Benchgen

No Benchgen results yet — be the first to run SkillsBench.

SkillsBench vs Other Benchmarks

SkillsBench is the most targeted benchmark for evaluating a single harness-design pattern — Skills/context packages — in isolation, whereas Harness-Bench and EnterpriseClawBench evaluate the harness as a whole. Use SkillsBench when deciding whether to invest in curated Skills authoring versus letting agents self-generate context.

Run SkillsBench on Your Model

Benchgen lets teams test whether curated Skills, self-generated Skills, or no Skills produces the best resolution rate on their own harness and task distribution — turning SkillsBench's paired methodology into a repeatable check before shipping a Skills-based context strategy to production.

Related Models & Benchmarks

• Related benchmarks: Harness-Bench · MCP Atlas · EnterpriseClawBench

Frequently Asked Questions