# XSTest (Harm) ![Category](https://img.shields.io/badge/Category-Safety-1f6feb) ![Metric](https://img.shields.io/badge/Metric-F1%20Score-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-High-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv%202308.01263-b31b1b)](https://arxiv.org/abs/2308.01263) [![GitHub](https://img.shields.io/badge/GitHub-paul--rottger%2Fexaggerated--safety-181717?logo=github&logoColor=white)](https://github.com/paul-rottger/exaggerated-safety) > **Quick answer:** XSTest (Harm) measures whether a classifier correctly judges the harmfulness of prompts in a 450-item suite (250 clearly safe, 200 clearly unsafe) built to test for both under- and over-cautious safety behavior. Because many "safe" prompts deliberately resemble unsafe ones in surface language, this is a strong test of whether a classifier is pattern-matching keywords or genuinely reasoning about intent. ## At a Glance **What it tests:** Whether a classifier correctly labels prompts as harmful or safe, including "trap" prompts that use sensitive-sounding language (e.g., discussing violence in a historical or medical context) but are not actually harmful. **Why it matters:** XSTest was purpose-built to expose exaggerated safety behavior — models and classifiers that over-flag benign content because it superficially resembles unsafe content. A classifier that scores well here demonstrates it isn't just keyword-matching. **Known limitations:** Small test set (450 items) relative to other guard benchmarks; English-only; the "safe" prompt set covers 10 specific prompt types rather than an exhaustive sampling of benign language. ## What XSTest (Harm) Measures XSTest comprises 250 safe prompts across 10 prompt types that well-calibrated systems should not flag, plus 200 unsafe prompts as contrasts that should be flagged. The safe prompts are deliberately constructed to share surface-level features (violent verbs, drug names, etc.) with the unsafe set, so a classifier relying on shallow lexical cues will misclassify them. This task scores overall harm classification accuracy across both halves of the set. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Safety / exaggerated safety detection | | Metric | F1 score (%) | | Test set size | 450 (250 safe + 200 unsafe) | | Prompt types | 10 | | Created by | Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, Dirk Hovy | | Paper | [XSTest (arXiv 2308.01263)](https://arxiv.org/abs/2308.01263) | | GitHub | [paul-rottger/exaggerated-safety](https://github.com/paul-rottger/exaggerated-safety) | | Dataset | [walledai/XSTest](https://huggingface.co/datasets/walledai/XSTest) | | License | CC-BY-4.0 (prompts) | ## How XSTest (Harm) Is Scored Each of the 450 prompts' ground-truth safe/unsafe label is compared against a classifier's prediction and scored as F1. ## State-of-the-Art Results Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [GPT-OSS-Safeguard-20B](/models/openai/gpt-oss-safeguard-20b) | **93.8%** | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 2 | [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) | 93.5% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 3 | [Qwen3Guard-8B](/models/alibaba/qwen3guard-8b) | 92.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 4 | [LlamaGuard-4-12B](/models/meta/llamaguard-4-12b) | 89% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 5 | [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) | 86.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 6 | [ShieldGemma-9B](/models/google/shieldgemma-9b) | 80.6% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | *Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.* ## XSTest (Harm) on Benchgen > No Benchgen results yet — [be the first to run XSTest (Harm)](/benchmarks/independent-research/xstest-harm). ## XSTest (Harm) vs Other Benchmarks | Benchmark | What it tests | Test size | Saturation | |---|---|---|---| | **XSTest (Harm)** | Exaggerated-safety-resistant harm classification | 450 | High | | WildGuardTest (Prompt) | Prompt-harm classification | 1,725 | Medium | | XSTest (Refusal) | Over-refusal detection | 450 | Medium | ## Run XSTest (Harm) on Your Model Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's XSTest scores to compare against Shieldstral and other guard models. ## Related Models & Benchmarks - Models: [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) · [GPT-OSS-Safeguard-20B](/models/openai/gpt-oss-safeguard-20b) - Benchmarks: [XSTest (Refusal)](/benchmarks/independent-research/xstest-refusal) · [WildGuardTest (Prompt)](/benchmarks/allen-institute-for-ai/wildguardtest-prompt) ## Frequently Asked Questions

What is XSTest?

XSTest is a 450-prompt test suite (250 safe, 200 unsafe) designed to identify exaggerated safety behavior — cases where a classifier or model over-flags benign content that superficially resembles unsafe content.

Why does it matter that "safe" prompts resemble "unsafe" ones?

It tests whether a classifier reasons about actual intent and context or just pattern-matches surface-level keywords, which is a common failure mode in simpler moderation systems.

Who created XSTest?

Paul Röttger and colleagues, published in 2023.

--- *Last updated 2026-08-12.*

Rank	Model	Score
1	gpt-oss-safeguard-20b	93.8
2	shieldstral-1-0	93.5
3	qwen3guard-8b	92.9
4	llamaguard-4-12b	89
5	nemotron-3-5-content-safety-4b	86.9
6	shieldgemma-9b	80.6

XSTest (Harm) — Results

XSTest (Harm) — Leaderboard

XSTest (Harm) Leaderboard