HarmBench (Response) — Results
| Rank | Model | Score |
|---|---|---|
| 1 | gpt-oss-safeguard-20b | 88.2 |
| 2 | shieldstral-1-0 | 87 |
| 3 | qwen3guard-8b | 86.8 |
| 4 | nemotron-3-5-content-safety-4b | 85.3 |
| 5 | llamaguard-4-12b | 82.8 |
| 6 | shieldgemma-9b | 52.3 |
HarmBench (Response)
1 phaseActive
Center for AI Safety's standardized red-teaming benchmark, response-classification task. 400 harmful behaviors. Mazeika et al., 2024.
Overview
HarmBench (Response)
Quick answer: HarmBench (Response) measures whether a classifier correctly identifies a jailbroken model's response as harmful — i.e., whether an attack against a target model actually succeeded — using the same 400-behavior framework as HarmBench (Prompt), but judging outputs rather than requests.
At a Glance
What it tests: Whether a classifier can tell if a model's response to an adversarial prompt actually contains the requested harmful content (a successful jailbreak) vs. a refusal or non-compliant answer. Why it matters: Response classification is the core mechanism HarmBench uses to score attack success rate for red-teaming research — a classifier's accuracy here directly determines whether jailbreak research results are trustworthy. Known limitations: Judging "did the response actually help" is harder and noisier than judging prompt intent, since responses vary widely in fluency, evasiveness, and partial compliance.
What HarmBench (Response) Measures
Using the same 400 standardized harmful behaviors, this task scores whether a classifier correctly labels a model's response as containing genuinely harmful, actionable content versus a refusal, deflection, or unhelpful non-compliant answer — the same underlying judgment HarmBench uses internally to compute attack success rates for red-teaming methods.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / red-teaming |
| Metric | F1 score (%) |
| Behaviors | 400 standardized harmful behaviors |
| Created by | Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, et al. (Center for AI Safety) |
| Paper | HarmBench (arXiv 2402.04249) |
| GitHub | centerforaisafety/HarmBench |
| License | MIT (framework); behaviors dataset under research-use terms |
How HarmBench (Response) Is Scored
Each response to a behavior-derived prompt is labeled harmful/not-harmful by human or LLM-judge annotation; a classifier's predictions are compared against ground truth and scored as F1.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%).
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | GPT-OSS-Safeguard-20B | 88.2% | Mistral AI model card | 2026-08 |
| 2 | Shieldstral 1.0 | 87% | Mistral AI model card | 2026-08 |
| 3 | Qwen3Guard-8B | 86.8% | Mistral AI model card | 2026-08 |
| 4 | Nemotron-3.5-Content-Safety-4B | 85.3% | Mistral AI model card | 2026-08 |
| 5 | LlamaGuard-4-12B | 82.8% | Mistral AI model card | 2026-08 |
| 6 | ShieldGemma-9B | 52.3% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.
HarmBench (Response) on Benchgen
No Benchgen results yet — be the first to run HarmBench (Response).
HarmBench (Response) vs Other Benchmarks
| Benchmark | What it tests | Test size | Saturation |
|---|---|---|---|
| HarmBench (Response) | Jailbreak success detection | 400 | Medium |
| WildGuardTest (Response) | Response-harm classification | 1,725 | Medium |
| BeaverTails | QA-pair harm classification | ~700 | Medium |
Run HarmBench (Response) on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's HarmBench response scores to compare against Shieldstral and other guard models.
Related Models & Benchmarks
- Models: Shieldstral 1.0 · Qwen3Guard-8B
- Benchmarks: HarmBench (Prompt) · BeaverTails
Frequently Asked Questions
What is HarmBench (Response)?
It's the response-classification task on the Center for AI Safety's HarmBench framework, judging whether a model's answer to an adversarial prompt actually contains harmful, actionable content.How does this relate to jailbreak "attack success rate"?
HarmBench uses this same classification judgment internally to compute attack success rates for red-teaming methods — a more accurate classifier here means more trustworthy jailbreak research results.Who created HarmBench?
Researchers at the Center for AI Safety, UIUC, and UC Berkeley (Mazeika et al., ICML 2024).Last updated 2026-08-12.