# LlavaGuard ![Category](https://img.shields.io/badge/Category-Multimodal%20Safety-1f6feb) ![Metric](https://img.shields.io/badge/Metric-F1%20Score-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv%202406.05113-b31b1b)](https://arxiv.org/abs/2406.05113) [![GitHub](https://img.shields.io/badge/GitHub-ml--research%2FLlavaGuard-181717?logo=github&logoColor=white)](https://github.com/ml-research/LlavaGuard) > **Quick answer:** LlavaGuard measures fine-grained, policy-based multimodal safety classification — going beyond a binary safe/unsafe label to judge image content against a detailed, customizable safety policy, similar in spirit to Shieldstral's own policy-adaptive approach but purpose-built for vision-language moderation. ## At a Glance **What it tests:** Whether a classifier can apply a detailed, structured safety policy (rather than a fixed category label) to image content, judging compliance the way a human moderator using written guidelines would. **Why it matters:** LlavaGuard's policy-driven format anticipated the kind of flexible, instruction-conditioned moderation that Shieldstral itself is built around — making this benchmark a natural point of comparison for policy-adaptive safety classifiers on images. **Known limitations:** Smaller-scale academic release relative to the large industrial guard datasets (e.g., Aegis v2, WildGuardMix); exact held-out test set size is not broken out precisely in public third-party comparisons. ## What LlavaGuard Measures LlavaGuard evaluates vision-language models against a structured safety taxonomy, judging image content according to a detailed written policy (categories, rationale, and safety rating) rather than a simple binary label — testing whether a classifier's judgment aligns with policy-grounded human annotation on visual content. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Safety / multimodal policy-based moderation | | Metric | F1 score (%) | | Modality | Image + policy text | | Created by | Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, Patrick Schramowski | | Affiliation | TU Darmstadt, hessian.AI | | Paper | [LlavaGuard (arXiv 2406.05113)](https://arxiv.org/abs/2406.05113) | | GitHub | [ml-research/LlavaGuard](https://github.com/ml-research/LlavaGuard) | | Dataset | [AIML-TUDA/LlavaGuard](https://huggingface.co/datasets/AIML-TUDA/LlavaGuard) | ## How LlavaGuard Is Scored Each image-policy pair's ground-truth safety judgment is compared against a classifier's prediction and scored as F1. ## State-of-the-Art Results Scores from the Shieldstral model card (Mistral AI, August 2026). F1 (%). Only models with native multimodal support report scores here. | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [LlavaGuard-7B](/models/tu-darmstadt-hessian-ai/llavaguard-7b) | **81.4%** | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 2 | [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) | 72.0% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 3 | [OmniGuard-7B](/models/independent-research/omniguard-7b) | 71.7% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 4 | [Nemotron-3.5-Content-Safety-4B](/models/nvidia/nemotron-3-5-content-safety-4b) | 70.0% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 5 | [ShieldGemma-2-4B](/models/google/shieldgemma-2-4b) | 56.2% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | | 6 | [LlamaGuard-4-12B](/models/meta/llamaguard-4-12b) | 21.9% | [Mistral AI model card](https://huggingface.co/mistralai/Shieldstral-1.0-3B) | 2026-08 | *Scores sourced from Mistral AI's published model card, shown for context. Not Benchgen measurements. LlavaGuard-7B naturally leads on this vendor-adjacent evaluation, being purpose-trained on this dataset's methodology; text-only guard models do not report scores here.* ## LlavaGuard on Benchgen > No Benchgen results yet — [be the first to run LlavaGuard](/benchmarks/tu-darmstadt-hessian-ai/llavaguard). ## LlavaGuard vs Other Benchmarks | Benchmark | What it tests | Modality | Saturation | |---|---|---|---| | **LlavaGuard** | Policy-based multimodal safety judgment | Image + policy text | Medium | | VLGuard | Image-instruction pair safety | Image + text | High | | UnsafeBench | Standalone unsafe image classification | Image only | Medium | ## Run LlavaGuard on Your Model Benchgen tracks version-controlled, regression-tested guard-model performance — submit your classifier's LlavaGuard scores to compare against Shieldstral and other multimodal guard models. ## Related Models & Benchmarks - Models: [LlavaGuard-7B](/models/tu-darmstadt-hessian-ai/llavaguard-7b) · [Shieldstral 1.0](/models/mistral-ai/shieldstral-1-0) - Benchmarks: [VLGuard](/benchmarks/independent-research/vlguard) · [UnsafeBench](/benchmarks/independent-research/unsafebench) ## Frequently Asked Questions

What is LlavaGuard?

LlavaGuard is a vision-language safety benchmark that judges image content against a detailed, structured safety policy rather than a simple binary label.

How does this compare to Shieldstral's own approach?

Both use policy-conditioned judgments rather than fixed categories, though LlavaGuard was purpose-built for image moderation specifically, while Shieldstral applies the same idea across text, image, and combined moderation.

Who created LlavaGuard?

Researchers at TU Darmstadt and hessian.AI (Helff et al., 2024).

--- *Last updated 2026-08-12.*

Rank	Model	Score
1	llavaguard-7b	81.4
2	shieldstral-1-0	72
3	omniguard-7b	71.7
4	nemotron-3-5-content-safety-4b	70
5	shieldgemma-2-4b	56.2
6	llamaguard-4-12b	21.9

LlavaGuard — Results

LlavaGuard — Phases

LlavaGuard