# V* ![Category](https://img.shields.io/badge/Category-Multimodal-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Accuracy-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-191-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-Dec%202023-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2312.14135) [![GitHub](https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white)](https://github.com/penghao-wu/vstar) > **Quick answer:** V* (V-Star) introduces an LLM-guided visual search mechanism for multimodal LLMs, and V*Bench is its companion benchmark of 191 high-resolution, visually crowded images testing whether models can precisely locate and reason about small visual details that standard MLLMs tend to overlook. ## At a Glance **What it tests:** Precise visual search in high-resolution, visually cluttered images — locating specific objects or attributes and reasoning about their relative positions — via direct-attribute and relative-position question types. **Why it matters:** Current MLLMs lack an explicit visual search mechanism, hindering their ability to focus on important visual details in crowded or high-resolution scenes; V*'s guided search mechanism (integrated into the "Show, sEArch, and TelL" or SEAL meta-architecture) directly addresses this gap and V*Bench measures the resulting improvement. **Known limitations:** V*Bench is a relatively small, targeted benchmark (191 images) focused specifically on visual search rather than broad multimodal understanding, so it complements rather than replaces larger general-purpose multimodal benchmarks. ## What V* Measures V* introduces an LLM-guided visual search mechanism that employs the world knowledge embedded in LLMs to perform efficient visual querying — essentially teaching a multimodal model to actively search for relevant image regions rather than passively processing the entire image at once. Combined with an MLLM, this mechanism enhances collaborative reasoning, contextual understanding, and precise targeting of specific visual elements, resulting in a new meta-architecture the authors call SEAL (Show, sEArch, and TelL). V*Bench, the benchmark created to evaluate this capability, is specifically designed to test MLLMs' ability to process high-resolution images and focus on visual details that are easy to miss without an explicit search step. Questions are split across direct-attribute recognition (identifying a specific detail directly) and relative-position reasoning (understanding spatial relationships between objects), both requiring the model to locate small or subtle visual elements within a larger, cluttered scene. The paper's results highlight the necessity of incorporating visual search capabilities into multimodal systems, showing that models augmented with V*'s guided search substantially outperform standard MLLMs lacking this mechanism on V*Bench's targeted, high-resolution test cases. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Multimodal (guided visual search / high-resolution detail recognition) | | Metric | Accuracy (%) across direct-attribute and relative-position question types | | Number of tasks | 191 high-resolution, visually crowded images | | Saturation | Low — standard MLLMs without explicit visual search mechanisms struggle | | Created by | Penghao Wu, Saining Xie | | Source paper | [Wu & Xie 2023](https://arxiv.org/abs/2312.14135) | | Code | [GitHub](https://github.com/penghao-wu/vstar) | | Dataset | [Hugging Face](https://huggingface.co/datasets/craigwu/vstar_bench) | ## How V* Is Scored Each question is scored as multiple-choice accuracy against the ground-truth answer, reported separately for the direct-attribute and relative-position subtasks as well as an overall V*Bench accuracy. ## State-of-the-Art Results > Scores vary by whether a model incorporates an explicit visual search mechanism (like SEAL) versus relying on standard MLLM processing alone. See the [project page](https://vstar-seal.github.io/) for current standings. ## V* on Benchgen > No Benchgen results yet — [be the first to run V*](/benchmarks/penghao-wu-saining-xie-new-york-university/v). ## V* vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **V*** | Guided visual search in high-resolution, crowded images | 191 | low | | [MME-RealWorld](/benchmarks/yi-fan-zhang-huanyu-zhang-haochen-tian-chaoyou-fu-shuangqing-zhang-et-al-xiamen-university-tencent-youtu-lab-chinese-academy-of-sciences/mme-realworld) | High-resolution real-world perception across 5 scenarios | 29,429 | low | | [VLMsAreBlind](/benchmarks/pooyan-rahmanzadehgervi-logan-bolton-mohammad-reza-taesiri-anh-totti-nguyen-auburn-university-university-of-alberta/vlmsareblind) | Low-level spatial perception (counting, intersections) | 7 | low | V* targets the specific challenge of finding small details in cluttered, high-resolution images, complementing VLMsAreBlind's focus on basic geometric perception and MME-RealWorld's broader real-world scenario coverage. ## Run V* on Your Model Benchgen lets teams run V*Bench against their own model or agent versions on demand, tracking accuracy on direct-attribute and relative-position questions to catch regressions in high-resolution visual search capability. ## Related Models & Benchmarks Explore related multimodal benchmarks like [MME-RealWorld](/benchmarks/yi-fan-zhang-huanyu-zhang-haochen-tian-chaoyou-fu-shuangqing-zhang-et-al-xiamen-university-tencent-youtu-lab-chinese-academy-of-sciences/mme-realworld) and [VLMsAreBlind](/benchmarks/pooyan-rahmanzadehgervi-logan-bolton-mohammad-reza-taesiri-anh-totti-nguyen-auburn-university-university-of-alberta/vlmsareblind) to compare high-resolution visual search with broader real-world perception and low-level spatial acuity evaluation. ## Frequently Asked Questions

What is V*?

V* is an LLM-guided visual search mechanism for multimodal LLMs, and V*Bench is its 191-image companion benchmark testing whether models can locate small visual details in high-resolution, visually crowded scenes.

What does a good V*Bench score look like?

Models augmented with an explicit visual search mechanism (like the paper's SEAL architecture) substantially outperform standard MLLMs on V*Bench, so a strong result shows consistent accuracy gains across both the direct-attribute and relative-position subtasks.

Who created V*?

V* and V*Bench were created by Penghao Wu and Saining Xie at New York University.

Rank	Model	Score
1	gpt-4v	54.97
2	gemini-pro	48.16

V* — Results

V* — Leaderboard

V* Leaderboard