# RTP-LX Prompt ![Category](https://img.shields.io/badge/Category-Multilingual%20Safety-1f6feb) ![Metric](https://img.shields.io/badge/Metric-F1%20Score-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv%202404.14397-b31b1b)](https://arxiv.org/abs/2404.14397) [![GitHub](https://img.shields.io/badge/GitHub-microsoft%2FRTP--LX-181717?logo=github&logoColor=white)](https://github.com/microsoft/RTP-LX) > **Quick answer:** RTP-LX Prompt measures multilingual toxicity classification across roughly 28 languages, using Microsoft's human-transcreated (not just translated) extension of the original RealToxicityPrompts benchmark. It's a relative weak spot for Shieldstral, where a larger competing model leads by double digits. ## At a Glance **What it tests:** Whether a classifier correctly flags toxic prompts across a broad set of languages, including many outside the typical high-resource set covered by other multilingual safety benchmarks. **Why it matters:** RTP-LX uses human "transcreation" (culturally adapted translation, not literal machine translation) so toxicity patterns reflect real linguistic and cultural nuance rather than translation artifacts. **Known limitations:** Per-language subset sizes vary; some included languages have comparatively little training data available for classifiers to learn from, which can depress aggregate scores. ## What RTP-LX Prompt Measures RTP-LX extends the RealToxicityPrompts methodology to roughly 28 languages via human transcreation and annotation, rather than relying purely on machine translation. This task scores whether a classifier's prompt-toxicity judgment holds up across that broader, culturally-adapted multilingual distribution. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Safety / multilingual toxicity | | Metric | F1 score (%) | | Languages | ~28 | | Created by | Adrian de Wynter, Ishaan Watts, Nektaria Potha, and colleagues (Microsoft) | | Paper | [RTP-LX (arXiv 2404.14397)](https://arxiv.org/abs/2404.14397) | | GitHub | [microsoft/RTP-LX](https://github.com/microsoft/RTP-LX) | | License | Research use (see repository terms) | ## How RTP-LX Prompt Is Scored Each prompt's ground-truth toxicity label is compared against a classifier's prediction and scored as F1, aggregated across the covered languages. ## State-of-the-Art Results Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) | **86.1%** | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 2 | [GPT-OSS-Safeguard-20B](/models/openai/gpt-oss-safeguard-20b) | 83.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 3 | [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) | 70.3% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 4 | [Qwen3Guard-8B](/models/alibaba/qwen3guard-8b) | 67.3% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 5 | [LlamaGuard-4-12B](/models/meta/llamaguard-4-12b) | 43.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 6 | [ShieldGemma-9B](/models/google/shieldgemma-9b) | 36.7% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | *Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. This is one of the benchmarks where Shieldstral's model card shows a notable gap to the leading model.* ## RTP-LX Prompt on Benchgen > No Benchgen results yet — [be the first to run RTP-LX Prompt](/benchmarks/microsoft/rtp-lx-prompt). ## RTP-LX Prompt vs Other Benchmarks | Benchmark | What it tests | Languages | Saturation | |---|---|---|---| | **RTP-LX Prompt** | Multilingual toxicity classification | ~28 | Medium | | PolyGuard Prompt | Multilingual prompt-harm classification | 17 | Medium | ## Run RTP-LX Prompt on Your Model Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's RTP-LX scores to compare against Shieldstral and other guard models. ## Related Models & Benchmarks - Models: [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) · [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) - Benchmarks: [RTP-LX Completion](/benchmarks/microsoft/rtp-lx-completion) · [PolyGuard Prompt](/benchmarks/toxicityprompts/polyguard-prompt) ## Frequently Asked Questions

What is RTP-LX?

RTP-LX is Microsoft's multilingual extension of the RealToxicityPrompts benchmark, covering roughly 28 languages via human transcreation rather than literal machine translation.

What does "transcreation" mean?

Transcreation adapts content to be culturally and linguistically appropriate in the target language rather than translating it literally — intended to preserve the intent and severity of toxic content across languages.

Who created RTP-LX?

Researchers at Microsoft (de Wynter et al., 2024).

--- *Last updated 2026-08-12.*

Rank	Model	Score
1	nemotron-3-5-content-safety-4b	86.1
2	gpt-oss-safeguard-20b	83.9
3	shieldstral-1-0	70.3
4	qwen3guard-8b	67.3
5	llamaguard-4-12b	43.9
6	shieldgemma-9b	36.7

RTP-LX Prompt — Results

RTP-LX Prompt — Evaluations