Aegis v2 (Prompt) — Results
| Rank | Model | Score |
|---|---|---|
| 1 | nemotron-3-5-content-safety-4b | 86.3 |
| 2 | shieldstral-1-0 | 86.2 |
| 3 | qwen3guard-8b | 84.6 |
| 4 | gpt-oss-safeguard-20b | 84.4 |
| 5 | llamaguard-4-12b | 71.5 |
| 6 | shieldgemma-9b | 65.8 |
Aegis v2 (Prompt)
1 phaseActive
NVIDIA's Nemotron Content Safety Dataset V2 (formerly Aegis 2.0), prompt-classification task. 1,964-item test set, 12-category taxonomy. Ghosh et al., 2025.
Overview
Aegis v2 (Prompt)
Quick answer: Aegis v2 (Prompt) measures prompt-harm classification against NVIDIA's 12-category (plus 9 fine-grained subcategory) safety taxonomy, using a 1,964-item held-out test set from the Nemotron Content Safety Dataset V2 (formerly Aegis AI Content Safety Dataset 2.0).
At a Glance
What it tests: Whether a classifier correctly flags harmful user prompts across a broad, fine-grained hazard taxonomy spanning hate speech, self-harm, weapons, criminal planning, and more. Why it matters: Aegis v2's taxonomy is one of the more comprehensive in guard-model literature (12 core + 9 fine-grained categories), and its responses were generated by Mistral-7B-v0.1 specifically because that model has low built-in refusal rates — producing genuinely harmful completions to label rather than safety-filtered ones. Known limitations: Primarily English; response data generated by a single base model (Mistral-7B-v0.1) rather than sourced from diverse production traffic.
What Aegis v2 (Prompt) Measures
The Nemotron Content Safety Dataset V2 contains 33,416 annotated human–LLM interactions (30,007 train / 1,445 validation / 1,964 test), hybrid-labeled via human annotation with LLM-jury augmentation for response labels. Aegis v2 (Prompt) scores the prompt-harm label specifically — classifying whether the initial user request violates any of the taxonomy's hazard categories, independent of the response.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / content moderation |
| Metric | F1 score (%) |
| Test set size | 1,964 items |
| Taxonomy | 12 core + 9 fine-grained hazard categories |
| Created by | Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea, Jibin Rajan Varghese, Christopher Parisien (NVIDIA) |
| Paper | AEGIS2.0 (NAACL 2025) |
| Dataset | nvidia/Aegis-AI-Content-Safety-Dataset-2.0 |
| License | CC-BY-4.0 |
How Aegis v2 (Prompt) Is Scored
Each of the 1,964 test items' human-annotated prompt-harm label is compared against a classifier's prediction, scored as F1.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%).
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Nemotron-3.5-Content-Safety-4B | 86.3% | Mistral AI model card | 2026-08 |
| 2 | Shieldstral 1.0 | 86.2% | Mistral AI model card | 2026-08 |
| 3 | Qwen3Guard-8B | 84.6% | Mistral AI model card | 2026-08 |
| 4 | GPT-OSS-Safeguard-20B | 84.4% | Mistral AI model card | 2026-08 |
| 5 | LlamaGuard-4-12B | 71.5% | Mistral AI model card | 2026-08 |
| 6 | ShieldGemma-9B | 65.8% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.
Aegis v2 (Prompt) on Benchgen
No Benchgen results yet — be the first to run Aegis v2 (Prompt).
Aegis v2 (Prompt) vs Other Benchmarks
| Benchmark | What it tests | Test size | Saturation |
|---|---|---|---|
| Aegis v2 (Prompt) | Fine-grained taxonomy prompt classification | 1,964 | Medium |
| WildGuardTest (Prompt) | Prompt-harm classification | 1,725 | Medium |
| HarmBench (Prompt) | Adversarial jailbreak prompt classification | 400 | Medium |
Run Aegis v2 (Prompt) on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's Aegis v2 scores to compare against Shieldstral and other guard models.
Related Models & Benchmarks
- Models: Shieldstral 1.0 · Nemotron-3.5-Content-Safety-4B
- Benchmarks: Aegis v2 (Response) · WildGuardTest (Prompt)
Frequently Asked Questions
What is Aegis v2?
Aegis v2 (now called the Nemotron Content Safety Dataset V2) is NVIDIA's safety classification dataset spanning a 12-category, 9-subcategory hazard taxonomy, with prompt and response classification tasks scored separately.Why was it renamed from "Aegis" to "Nemotron Content Safety Dataset"?
NVIDIA consolidated its safety datasets under the Nemotron branding; the underlying dataset and taxonomy are unchanged.Who created Aegis v2?
NVIDIA's NeMo Guardrails team (Ghosh et al.), published at NAACL 2025.Last updated 2026-08-12.