XSTest (Harm) — Results
| Rank | Model | Score |
|---|---|---|
| 1 | gpt-oss-safeguard-20b | 93.8 |
| 2 | shieldstral-1-0 | 93.5 |
| 3 | qwen3guard-8b | 92.9 |
| 4 | llamaguard-4-12b | 89 |
| 5 | nemotron-3-5-content-safety-4b | 86.9 |
| 6 | shieldgemma-9b | 80.6 |
XSTest (Harm)
1 phaseActive
A 450-prompt test suite (250 safe, 200 unsafe) for identifying exaggerated safety behaviors. Röttger et al., 2023.
Overview
XSTest (Harm)
Quick answer: XSTest (Harm) measures whether a classifier correctly judges the harmfulness of prompts in a 450-item suite (250 clearly safe, 200 clearly unsafe) built to test for both under- and over-cautious safety behavior. Because many "safe" prompts deliberately resemble unsafe ones in surface language, this is a strong test of whether a classifier is pattern-matching keywords or genuinely reasoning about intent.
At a Glance
What it tests: Whether a classifier correctly labels prompts as harmful or safe, including "trap" prompts that use sensitive-sounding language (e.g., discussing violence in a historical or medical context) but are not actually harmful. Why it matters: XSTest was purpose-built to expose exaggerated safety behavior — models and classifiers that over-flag benign content because it superficially resembles unsafe content. A classifier that scores well here demonstrates it isn't just keyword-matching. Known limitations: Small test set (450 items) relative to other guard benchmarks; English-only; the "safe" prompt set covers 10 specific prompt types rather than an exhaustive sampling of benign language.
What XSTest (Harm) Measures
XSTest comprises 250 safe prompts across 10 prompt types that well-calibrated systems should not flag, plus 200 unsafe prompts as contrasts that should be flagged. The safe prompts are deliberately constructed to share surface-level features (violent verbs, drug names, etc.) with the unsafe set, so a classifier relying on shallow lexical cues will misclassify them. This task scores overall harm classification accuracy across both halves of the set.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / exaggerated safety detection |
| Metric | F1 score (%) |
| Test set size | 450 (250 safe + 200 unsafe) |
| Prompt types | 10 |
| Created by | Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, Dirk Hovy |
| Paper | XSTest (arXiv 2308.01263) |
| GitHub | paul-rottger/exaggerated-safety |
| Dataset | walledai/XSTest |
| License | CC-BY-4.0 (prompts) |
How XSTest (Harm) Is Scored
Each of the 450 prompts' ground-truth safe/unsafe label is compared against a classifier's prediction and scored as F1.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%).
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | GPT-OSS-Safeguard-20B | 93.8% | Mistral AI model card | 2026-08 |
| 2 | Shieldstral 1.0 | 93.5% | Mistral AI model card | 2026-08 |
| 3 | Qwen3Guard-8B | 92.9% | Mistral AI model card | 2026-08 |
| 4 | LlamaGuard-4-12B | 89% | Mistral AI model card | 2026-08 |
| 5 | Nemotron-3.5-Content-Safety-4B | 86.9% | Mistral AI model card | 2026-08 |
| 6 | ShieldGemma-9B | 80.6% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.
XSTest (Harm) on Benchgen
No Benchgen results yet — be the first to run XSTest (Harm).
XSTest (Harm) vs Other Benchmarks
| Benchmark | What it tests | Test size | Saturation |
|---|---|---|---|
| XSTest (Harm) | Exaggerated-safety-resistant harm classification | 450 | High |
| WildGuardTest (Prompt) | Prompt-harm classification | 1,725 | Medium |
| XSTest (Refusal) | Over-refusal detection | 450 | Medium |
Run XSTest (Harm) on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's XSTest scores to compare against Shieldstral and other guard models.
Related Models & Benchmarks
- Models: Shieldstral 1.0 · GPT-OSS-Safeguard-20B
- Benchmarks: XSTest (Refusal) · WildGuardTest (Prompt)
Frequently Asked Questions
What is XSTest?
XSTest is a 450-prompt test suite (250 safe, 200 unsafe) designed to identify exaggerated safety behavior — cases where a classifier or model over-flags benign content that superficially resembles unsafe content.Why does it matter that "safe" prompts resemble "unsafe" ones?
It tests whether a classifier reasons about actual intent and context or just pattern-matches surface-level keywords, which is a common failure mode in simpler moderation systems.Who created XSTest?
Paul Röttger and colleagues, published in 2023.Last updated 2026-08-12.