UnsafeBench — Results
| Rank | Model | Score |
|---|---|---|
| 1 | shieldstral-1-0 | 81.8 |
| 2 | omniguard-7b | 72.6 |
| 3 | nemotron-3-5-content-safety-4b | 67.7 |
| 4 | llavaguard-7b | 63.9 |
| 5 | shieldgemma-2-4b | 54.9 |
| 6 | llamaguard-4-12b | 30.8 |
UnsafeBench
1 phaseActive
A 10,000-image benchmark for evaluating unsafe image classification, spanning real and AI-generated content. Qu et al., 2024.
Overview
UnsafeBench
Quick answer: UnsafeBench measures standalone image safety classification — no accompanying text required — across a curated set of roughly 10,000 real-world and AI-generated images. It tests whether a classifier can flag unsafe visual content on its own, independent of any caption or instruction.
At a Glance
What it tests: Whether a classifier can correctly label an image alone as safe or unsafe, across a mix of real photographs and images generated by text-to-image models. Why it matters: As AI image generation becomes widespread, moderating generated content requires classifiers that generalize across both real-world and synthetic image distributions — UnsafeBench was built specifically to test that generalization. Known limitations: Curated dataset rather than production-traffic-derived; category boundaries for "unsafe" content reflect the curators' taxonomy choices.
What UnsafeBench Measures
UnsafeBench combines roughly 10,000 real and AI-generated images labeled for safety, testing image-only classification without any accompanying instruction or caption — a purer test of visual harm detection than instruction-paired benchmarks like VLGuard.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Safety / image content moderation |
| Metric | F1 score (%) |
| Test set size | ~10,000 images |
| Modality | Image only |
| Created by | Yiting Qu and colleagues |
| Paper | UnsafeBench (arXiv 2405.03486) |
| GitHub | YitingQu/unsafe-diffusion |
How UnsafeBench Is Scored
Each image's ground-truth safety label is compared against a classifier's prediction and scored as F1.
State-of-the-Art Results
Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). Only models with native multimodal support report scores here.
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Shieldstral 1.0 | 81.8% | Mistral AI model card | 2026-08 |
| 2 | OmniGuard-7B | 72.6% | Mistral AI model card | 2026-08 |
| 3 | Nemotron-3.5-Content-Safety-4B | 67.7% | Mistral AI model card | 2026-08 |
| 4 | LlavaGuard-7B | 63.9% | Mistral AI model card | 2026-08 |
| 5 | ShieldGemma-2-4B | 54.9% | Mistral AI model card | 2026-08 |
| 6 | LlamaGuard-4-12B | 30.8% | Mistral AI model card | 2026-08 |
Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. Text-only guard models do not report scores on this multimodal benchmark.
UnsafeBench on Benchgen
No Benchgen results yet — be the first to run UnsafeBench.
UnsafeBench vs Other Benchmarks
| Benchmark | What it tests | Modality | Saturation |
|---|---|---|---|
| UnsafeBench | Standalone unsafe image classification | Image only | Medium |
| VLGuard | Image-instruction pair safety | Image + text | High |
| LlavaGuard | Fine-grained multimodal policy safety | Image + text | Medium |
Run UnsafeBench on Your Model
Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's UnsafeBench scores to compare against Shieldstral and other multimodal guard models.
Related Models & Benchmarks
- Models: Shieldstral 1.0 · LlavaGuard-7B
- Benchmarks: VLGuard · LlavaGuard
Frequently Asked Questions
What is UnsafeBench?
UnsafeBench is an image safety benchmark of roughly 10,000 real and AI-generated images, testing standalone image classification without accompanying text.How does this differ from VLGuard?
UnsafeBench classifies images alone; VLGuard classifies image-instruction pairs, testing whether harm arises from the combination of visual and textual content.Who created UnsafeBench?
Yiting Qu and colleagues, published in 2024.Last updated 2026-08-12.