RTP-LX Prompt — Results
| Rank | Model | Score |
|---|---|---|
| 1 | nemotron-3-5-content-safety-4b | 86.1 |
| 2 | gpt-oss-safeguard-20b | 83.9 |
| 3 | shieldstral-1-0 | 70.3 |
| 4 | qwen3guard-8b | 67.3 |
| 5 | llamaguard-4-12b | 43.9 |
| 6 | shieldgemma-9b | 36.7 |
RTP-LX Prompt
1 phaseActive
Microsoft's multilingual extension of RealToxicityPrompts, spanning 28 languages, prompt-classification task. de Wynter et al., 2024.
Overview
RTP-LX Prompt
Quick answer: RTP-LX Prompt measures multilingual toxicity classification across roughly 28 languages, using Microsoft's human-transcreated (not just translated) extension of the original RealToxicityPrompts benchmark. It's a relative weak spot for Shieldstral, where a larger competing model leads by double digits.
At a Glance
What it tests: Whether a classifier correctly flags toxic prompts across a broad set of languages, including many outside the typical high-resource set covered by other multilingual safety benchmarks. Why it matters: RTP-LX uses human "transcreation" (culturally adapted translation, not literal machine translation) so toxicity patterns reflect real linguistic and cultural nuance rather than translation artifacts. Known limitations: Per-language subset sizes vary; some included languages have comparatively little training data available for classifiers to learn from, which can depress aggregate scores.
What RTP-LX Prompt Measures
RTP-LX extends the RealToxicityPrompts methodology to roughly 28 languages via human transcreation and annotation, rather than relying purely on machine translation. This task scores whether a classifier's prompt-toxicity judgment holds up across that broader, culturally-adapted multilingual distribution.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / multilingual toxicity |
| Metric | F1 score (%) |
| Languages | ~28 |
| Created by | Adrian de Wynter, Ishaan Watts, Nektaria Potha, and colleagues (Microsoft) |
| Paper | RTP-LX (arXiv 2404.14397) |
| GitHub | microsoft/RTP-LX |
| License | Research use (see repository terms) |
How RTP-LX Prompt Is Scored
Each prompt's ground-truth toxicity label is compared against a classifier's prediction and scored as F1, aggregated across the covered languages.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%).
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Nemotron-3.5-Content-Safety-4B | 86.1% | Mistral AI model card | 2026-08 |
| 2 | GPT-OSS-Safeguard-20B | 83.9% | Mistral AI model card | 2026-08 |
| 3 | Shieldstral 1.0 | 70.3% | Mistral AI model card | 2026-08 |
| 4 | Qwen3Guard-8B | 67.3% | Mistral AI model card | 2026-08 |
| 5 | LlamaGuard-4-12B | 43.9% | Mistral AI model card | 2026-08 |
| 6 | ShieldGemma-9B | 36.7% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. This is one of the benchmarks where Shieldstral's model card shows a notable gap to the leading model.
RTP-LX Prompt on Benchgen
No Benchgen results yet — be the first to run RTP-LX Prompt.
RTP-LX Prompt vs Other Benchmarks
| Benchmark | What it tests | Languages | Saturation |
|---|---|---|---|
| RTP-LX Prompt | Multilingual toxicity classification | ~28 | Medium |
| PolyGuard Prompt | Multilingual prompt-harm classification | 17 | Medium |
Run RTP-LX Prompt on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's RTP-LX scores to compare against Shieldstral and other guard models.
Related Models & Benchmarks
- Models: Nemotron-3.5-Content-Safety-4B · Shieldstral 1.0
- Benchmarks: RTP-LX Completion · PolyGuard Prompt
Frequently Asked Questions
What is RTP-LX?
RTP-LX is Microsoft's multilingual extension of the RealToxicityPrompts benchmark, covering roughly 28 languages via human transcreation rather than literal machine translation.What does "transcreation" mean?
Transcreation adapts content to be culturally and linguistically appropriate in the target language rather than translating it literally — intended to preserve the intent and severity of toxic content across languages.Who created RTP-LX?
Researchers at Microsoft (de Wynter et al., 2024).Last updated 2026-08-12.