# RefCOCO ![Category](https://img.shields.io/badge/Category-Reasoning-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Precision%401-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) ![Created](https://img.shields.io/badge/Created-2014-555) [![Paper](https://img.shields.io/badge/Paper-ACL%20Anthology-b31b1b)](https://aclanthology.org/D14-1086/) [![GitHub](https://img.shields.io/badge/GitHub-lichengunc%2Frefer-181717?logo=github&logoColor=white)](https://github.com/lichengunc/refer) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/jxu124/refcoco) > **Quick answer:** RefCOCO is a referring expression comprehension benchmark built on MS-COCO images, testing whether a model can locate the specific object in an image described by a natural-language expression (e.g., "the woman in the red hat"). Scores are typically reported as precision@1 averaged across the val, testA, and testB splits (and the related RefCOCO+ and RefCOCOg variants). LFM2.5-VL-3B averages 87.9% across these splits as of August 2026. ## At a Glance **What it tests:** A vision-language model's ability to ground a natural-language referring expression to the single correct object instance in an image, typically scored via bounding-box overlap or precision@1. **Why it matters:** Referring expression comprehension underlies visual grounding for tool use and robotics — an agent that needs to "click the second button from the left" or "pick up the blue mug" depends on the same grounding skill RefCOCO measures. **Known limitations:** RefCOCO is over a decade old and built on MS-COCO's relatively narrow object categories; frontier models now score in the high 80s–90s, making the benchmark less discriminating at the top end than newer grounding benchmarks. ## What RefCOCO Measures RefCOCO (and its variants RefCOCO+ and RefCOCOg) was introduced by Kazemzadeh et al. as a dataset of natural-language referring expressions collected via a two-player game, paired with MS-COCO images containing multiple instances of the same object category. A model must identify which specific instance the expression refers to — disambiguating between, for example, multiple people or multiple chairs in the same image. RefCOCO+ removes location-based expressions (forcing purely appearance-based descriptions), while RefCOCOg uses longer, more descriptive expressions collected in a non-interactive setting. Scores across all splits (val, testA, testB for RefCOCO/RefCOCO+; val, test for RefCOCOg) are commonly averaged into a single "RefCOCO-avg" headline metric, as reported by Liquid AI for LFM2.5-VL-3B. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Reasoning (visual grounding) | | Metric | Precision@1, averaged across RefCOCO/RefCOCO+/RefCOCOg splits | | Saturation | Medium | | Created by | Kazemzadeh et al. | | Source paper | [Kazemzadeh et al. 2014](https://aclanthology.org/D14-1086/) | | GitHub | [lichengunc/refer](https://github.com/lichengunc/refer) | | Dataset | [jxu124/refcoco on HuggingFace](https://huggingface.co/datasets/jxu124/refcoco) | ## How RefCOCO Is Scored For each referring expression, the model predicts a bounding box or point; the prediction is scored correct (precision@1) if it sufficiently overlaps with (or falls within) the ground-truth object's region. Per-split scores (val/testA/testB and the RefCOCO+/RefCOCOg equivalents) are commonly averaged into one headline number. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [LFM2.5-VL-3B](/models/liquid-ai/lfm2-5-vl-3b) | 87.9% (avg) | [LFM2.5-VL-3B launch blog](https://www.liquid.ai/blog/lfm2-5-vl-3b) | 2026-08 | *Score sourced from Liquid AI's LFM2.5-VL-3B announcement, August 2026.* ## RefCOCO on Benchgen > No Benchgen results yet — [be the first to run RefCOCO](/benchmarks/kazemzadeh-et-al/refcoco). ## RefCOCO vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **RefCOCO** | Referring expression comprehension (object grounding) | Medium | | [ScreenSpot-v2](/benchmarks/cheng-et-al/screenspot-v2) | GUI element grounding | Medium | | [BLINK](/benchmarks/fu-et-al/blink) | Multi-image visual perception | Low | ## Run RefCOCO on Your Model Benchgen lets you evaluate your own vision-language model's grounding accuracy and track it across model and version changes. ## Frequently Asked Questions

What is RefCOCO?

RefCOCO is a referring expression comprehension benchmark built on MS-COCO images, testing whether a model can locate the specific object an expression describes.

What does a good score look like on RefCOCO?

LFM2.5-VL-3B reports an 87.9% average across splits as of August 2026; frontier vision-language models typically score in the high 80s to low 90s.

Who created RefCOCO?

RefCOCO was introduced by Kazemzadeh et al. in 2014; see the [original paper](https://aclanthology.org/D14-1086/).

Is RefCOCO saturated?

It's approaching saturation for frontier models — scores in the high 80s to 90s are now common, making it less discriminating at the top end than newer grounding benchmarks.

What's the difference between RefCOCO, RefCOCO+, and RefCOCOg?

RefCOCO uses short expressions that may include location cues; RefCOCO+ forbids location-based expressions, forcing purely appearance-based descriptions; RefCOCOg uses longer, more descriptive expressions collected non-interactively.

Rank	Model	Score
1	lfm3-vl-2b	90.5
2	lfm2-5-vl-3b	87.9
3	north-micro-vision-instruct	73.2

RefCOCO — Results

RefCOCO — Leaderboard

RefCOCO Leaderboard