Qwen3GuardTest

Qwen3GuardTest

Quick answer: Qwen3GuardTest is Alibaba's held-out safety evaluation benchmark released alongside Qwen3Guard, used to score prompt and response safety classification. It's referenced by competing guard-model releases (including Shieldstral) as a comparison point outside of each vendor's own internal test sets.

At a Glance

What it tests: Safety classification accuracy against Alibaba's own evaluation methodology for the Qwen3Guard family of guard models. Why it matters: As a benchmark released alongside a specific guard model family, Qwen3GuardTest gives an independent-vendor cross-check — seeing how a competing classifier performs on Alibaba's own test set complements evaluation on more neutral, third-party benchmarks like WildGuardTest. Known limitations: Full test set composition and exact size are not broken out in public third-party comparisons; treat scores here as directionally useful rather than precisely reproducible without access to Alibaba's original evaluation harness.

What Qwen3GuardTest Measures

Qwen3Guard is Alibaba's guard-model family (available in Gen and Stream variants across multiple sizes) built for real-time and generation-time safety classification. Qwen3GuardTest is the evaluation methodology and held-out data referenced in the Qwen3Guard technical report, used both to validate Qwen3Guard itself and, increasingly, as a cross-vendor comparison benchmark for other safety classifiers such as Shieldstral.

Benchmark Specifications

How Qwen3GuardTest Is Scored

Classifiers are scored as F1 against Alibaba's held-out labeled evaluation set, following the methodology described in the Qwen3Guard technical report.

State-of-the-Art Results

Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). Qwen3Guard-8B figures reflect an average over strict/loose label mappings per the model card's methodology notes.

Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. Qwen3Guard-8B scores near the top here, as expected on its own vendor-released evaluation set — but GPT-OSS-Safeguard-20B edges narrowly ahead.

Qwen3GuardTest on Benchgen

No Benchgen results yet — be the first to run Qwen3GuardTest.

Qwen3GuardTest vs Other Benchmarks

Run Qwen3GuardTest on Your Model

Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's Qwen3GuardTest scores to compare against Shieldstral and other guard models.

Related Models & Benchmarks

• Models: Qwen3Guard-8B · Shieldstral 1.0
• Benchmarks: WildGuardTest (Prompt) · Aegis v2 (Prompt)

Frequently Asked Questions

Last updated 2026-08-12.