RTP-LX Completion — Results
| Rank | Model | Score |
|---|---|---|
| 1 | nemotron-3-5-content-safety-4b | 95.9 |
| 2 | gpt-oss-safeguard-20b | 94.6 |
| 3 | qwen3guard-8b | 93.9 |
| 4 | shieldstral-1-0 | 93.5 |
| 5 | shieldgemma-9b | 79 |
| 6 | llamaguard-4-12b | 66.5 |
RTP-LX Completion
1 phaseActive
Microsoft's multilingual extension of RealToxicityPrompts, spanning ~28 languages, completion-classification task. de Wynter et al., 2024.
Overview
RTP-LX Completion
Quick answer: RTP-LX Completion measures multilingual toxicity classification on model-generated completions (rather than prompts) across roughly 28 languages, using Microsoft's human-transcreated extension of RealToxicityPrompts.
At a Glance
What it tests: Whether a classifier correctly flags toxic model completions across a broad multilingual set, given a prompt that elicited the completion. Why it matters: Completion-level toxicity classification tests whether a model's actual generated text is toxic — a downstream signal that complements prompt-level filtering, particularly relevant for open-ended generation products deployed across many locales. Known limitations: Same per-language coverage caveats as RTP-LX Prompt — some languages have less representative data than others.
What RTP-LX Completion Measures
Using the same ~28-language RTP-LX corpus, this task scores classifier judgments on model completions rather than prompts — testing whether toxicity detection generalizes to generated text across the covered language set.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / multilingual toxicity |
| Metric | F1 score (%) |
| Languages | ~28 |
| Created by | Adrian de Wynter, Ishaan Watts, Nektaria Potha, and colleagues (Microsoft) |
| Paper | RTP-LX (arXiv 2404.14397) |
| GitHub | microsoft/RTP-LX |
| License | Research use (see repository terms) |
How RTP-LX Completion Is Scored
Each completion's ground-truth toxicity label is compared against a classifier's prediction and scored as F1, aggregated across the covered languages.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%).
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Nemotron-3.5-Content-Safety-4B | 95.9% | Mistral AI model card | 2026-08 |
| 2 | GPT-OSS-Safeguard-20B | 94.6% | Mistral AI model card | 2026-08 |
| 3 | Qwen3Guard-8B | 93.9% | Mistral AI model card | 2026-08 |
| 4 | Shieldstral 1.0 | 93.5% | Mistral AI model card | 2026-08 |
| 5 | ShieldGemma-9B | 79% | Mistral AI model card | 2026-08 |
| 6 | LlamaGuard-4-12B | 66.5% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements.
RTP-LX Completion on Benchgen
No Benchgen results yet — be the first to run RTP-LX Completion.
RTP-LX Completion vs Other Benchmarks
| Benchmark | What it tests | Languages | Saturation |
|---|---|---|---|
| RTP-LX Completion | Multilingual completion toxicity | ~28 | Medium |
| RTP-LX Prompt | Multilingual prompt toxicity | ~28 | Medium |
| PolyGuard Response | Multilingual response-harm classification | 17 | Medium |
Run RTP-LX Completion on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's RTP-LX completion scores to compare against Shieldstral and other guard models.
Related Models & Benchmarks
- Models: Shieldstral 1.0 · GPT-OSS-Safeguard-20B
- Benchmarks: RTP-LX Prompt · PolyGuard Response
Frequently Asked Questions
What is RTP-LX Completion?
It's the completion-toxicity classification task on Microsoft's RTP-LX benchmark, testing whether a classifier can flag toxic model-generated text across roughly 28 languages.How is this different from RTP-LX Prompt?
RTP-LX Prompt scores toxicity in the user's input; RTP-LX Completion scores toxicity in the model's generated output for that input.Who created RTP-LX?
Researchers at Microsoft (de Wynter et al., 2024).Last updated 2026-08-12.