# UnsafeBench ![Category](https://img.shields.io/badge/Category-Multimodal%20Safety-1f6feb) ![Metric](https://img.shields.io/badge/Metric-F1%20Score-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv%202405.03486-b31b1b)](https://arxiv.org/abs/2405.03486) [![GitHub](https://img.shields.io/badge/GitHub-YitingQu%2Funsafe--diffusion-181717?logo=github&logoColor=white)](https://github.com/YitingQu/unsafe-diffusion) > **Quick answer:** UnsafeBench measures standalone image safety classification — no accompanying text required — across a curated set of roughly 10,000 real-world and AI-generated images. It tests whether a classifier can flag unsafe visual content on its own, independent of any caption or instruction. ## At a Glance **What it tests:** Whether a classifier can correctly label an image alone as safe or unsafe, across a mix of real photographs and images generated by text-to-image models. **Why it matters:** As AI image generation becomes widespread, moderating generated content requires classifiers that generalize across both real-world and synthetic image distributions — UnsafeBench was built specifically to test that generalization. **Known limitations:** Curated dataset rather than production-traffic-derived; category boundaries for "unsafe" content reflect the curators' taxonomy choices. ## What UnsafeBench Measures UnsafeBench combines roughly 10,000 real and AI-generated images labeled for safety, testing image-only classification without any accompanying instruction or caption — a purer test of visual harm detection than instruction-paired benchmarks like VLGuard. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Safety / image content moderation | | Metric | F1 score (%) | | Test set size | ~10,000 images | | Modality | Image only | | Created by | Yiting Qu and colleagues | | Paper | [UnsafeBench (arXiv 2405.03486)](https://arxiv.org/abs/2405.03486) | | GitHub | [YitingQu/unsafe-diffusion](https://github.com/YitingQu/unsafe-diffusion) | ## How UnsafeBench Is Scored Each image's ground-truth safety label is compared against a classifier's prediction and scored as F1. ## State-of-the-Art Results Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). Only models with native multimodal support report scores here. | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) | **81.8%** | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 2 | [OmniGuard-7B](/models/independent-research/omniguard-7b) | 72.6% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 3 | [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) | 67.7% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 4 | [LlavaGuard-7B](/models/tu-darmstadt-hessian-ai/llavaguard-7b) | 63.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 5 | [ShieldGemma-2-4B](/models/google/shieldgemma-2-4b) | 54.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 6 | [LlamaGuard-4-12B](/models/meta/llamaguard-4-12b) | 30.8% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | *Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. Text-only guard models do not report scores on this multimodal benchmark.* ## UnsafeBench on Benchgen > No Benchgen results yet — [be the first to run UnsafeBench](/benchmarks/independent-research/unsafebench). ## UnsafeBench vs Other Benchmarks | Benchmark | What it tests | Modality | Saturation | |---|---|---|---| | **UnsafeBench** | Standalone unsafe image classification | Image only | Medium | | VLGuard | Image-instruction pair safety | Image + text | High | | LlavaGuard | Fine-grained multimodal policy safety | Image + text | Medium | ## Run UnsafeBench on Your Model Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's UnsafeBench scores to compare against Shieldstral and other multimodal guard models. ## Related Models & Benchmarks - Models: [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) · [LlavaGuard-7B](/models/tu-darmstadt-hessian-ai/llavaguard-7b) - Benchmarks: [VLGuard](/benchmarks/independent-research/vlguard) · [LlavaGuard](/benchmarks/tu-darmstadt-hessian-ai/llavaguard) ## Frequently Asked Questions

What is UnsafeBench?

UnsafeBench is an image safety benchmark of roughly 10,000 real and AI-generated images, testing standalone image classification without accompanying text.

How does this differ from VLGuard?

UnsafeBench classifies images alone; VLGuard classifies image-instruction pairs, testing whether harm arises from the combination of visual and textual content.

Who created UnsafeBench?

Yiting Qu and colleagues, published in 2024.

--- *Last updated 2026-08-12.*

Rank	Model	Score
1	shieldstral-1-0	81.8
2	omniguard-7b	72.6
3	nemotron-3-5-content-safety-4b	67.7
4	llavaguard-7b	63.9
5	shieldgemma-2-4b	54.9
6	llamaguard-4-12b	30.8

UnsafeBench — Results

UnsafeBench — Evaluations