# Aegis v2 (Prompt) ![Category](https://img.shields.io/badge/Category-Safety-1f6feb) ![Metric](https://img.shields.io/badge/Metric-F1%20Score-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv%202501.09004-b31b1b)](https://arxiv.org/abs/2501.09004) > **Quick answer:** Aegis v2 (Prompt) measures prompt-harm classification against NVIDIA's 12-category (plus 9 fine-grained subcategory) safety taxonomy, using a 1,964-item held-out test set from the Nemotron Content Safety Dataset V2 (formerly Aegis AI Content Safety Dataset 2.0). ## At a Glance **What it tests:** Whether a classifier correctly flags harmful user prompts across a broad, fine-grained hazard taxonomy spanning hate speech, self-harm, weapons, criminal planning, and more. **Why it matters:** Aegis v2's taxonomy is one of the more comprehensive in guard-model literature (12 core + 9 fine-grained categories), and its responses were generated by Mistral-7B-v0.1 specifically because that model has low built-in refusal rates — producing genuinely harmful completions to label rather than safety-filtered ones. **Known limitations:** Primarily English; response data generated by a single base model (Mistral-7B-v0.1) rather than sourced from diverse production traffic. ## What Aegis v2 (Prompt) Measures The Nemotron Content Safety Dataset V2 contains 33,416 annotated human–LLM interactions (30,007 train / 1,445 validation / 1,964 test), hybrid-labeled via human annotation with LLM-jury augmentation for response labels. Aegis v2 (Prompt) scores the prompt-harm label specifically — classifying whether the initial user request violates any of the taxonomy's hazard categories, independent of the response. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Safety / content moderation | | Metric | F1 score (%) | | Test set size | 1,964 items | | Taxonomy | 12 core + 9 fine-grained hazard categories | | Created by | Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea, Jibin Rajan Varghese, Christopher Parisien (NVIDIA) | | Paper | [AEGIS2.0 (NAACL 2025)](https://arxiv.org/abs/2501.09004) | | Dataset | [nvidia/Aegis-AI-Content-Safety-Dataset-2.0](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) | | License | CC-BY-4.0 | ## How Aegis v2 (Prompt) Is Scored Each of the 1,964 test items' human-annotated prompt-harm label is compared against a classifier's prediction, scored as F1. ## State-of-the-Art Results Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) | **86.3%** | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 2 | [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) | 86.2% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 3 | [Qwen3Guard-8B](/models/alibaba/qwen3guard-8b) | 84.6% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 4 | [GPT-OSS-Safeguard-20B](/models/openai/gpt-oss-safeguard-20b) | 84.4% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 5 | [LlamaGuard-4-12B](/models/meta/llamaguard-4-12b) | 71.5% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 6 | [ShieldGemma-9B](/models/google/shieldgemma-9b) | 65.8% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | *Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.* ## Aegis v2 (Prompt) on Benchgen > No Benchgen results yet — [be the first to run Aegis v2 (Prompt)](/benchmarks/nvidia/aegis-v2-prompt). ## Aegis v2 (Prompt) vs Other Benchmarks | Benchmark | What it tests | Test size | Saturation | |---|---|---|---| | **Aegis v2 (Prompt)** | Fine-grained taxonomy prompt classification | 1,964 | Medium | | WildGuardTest (Prompt) | Prompt-harm classification | 1,725 | Medium | | HarmBench (Prompt) | Adversarial jailbreak prompt classification | 400 | Medium | ## Run Aegis v2 (Prompt) on Your Model Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's Aegis v2 scores to compare against Shieldstral and other guard models. ## Related Models & Benchmarks - Models: [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) · [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) - Benchmarks: [Aegis v2 (Response)](/benchmarks/nvidia/aegis-v2-response) · [WildGuardTest (Prompt)](/benchmarks/allen-institute-for-ai/wildguardtest-prompt) ## Frequently Asked Questions

What is Aegis v2?

Aegis v2 (now called the Nemotron Content Safety Dataset V2) is NVIDIA's safety classification dataset spanning a 12-category, 9-subcategory hazard taxonomy, with prompt and response classification tasks scored separately.

Why was it renamed from "Aegis" to "Nemotron Content Safety Dataset"?

NVIDIA consolidated its safety datasets under the Nemotron branding; the underlying dataset and taxonomy are unchanged.

Who created Aegis v2?

NVIDIA's NeMo Guardrails team (Ghosh et al.), published at NAACL 2025.

--- *Last updated 2026-08-12.*

Rank	Model	Score
1	nemotron-3-5-content-safety-4b	86.3
2	shieldstral-1-0	86.2
3	qwen3guard-8b	84.6
4	gpt-oss-safeguard-20b	84.4
5	llamaguard-4-12b	71.5
6	shieldgemma-9b	65.8

Aegis v2 (Prompt) — Results

Aegis v2 (Prompt) — Leaderboard

Aegis v2 (Prompt) Leaderboard