RefCOCO

RefCOCO

Quick answer: RefCOCO is a referring expression comprehension benchmark built on MS-COCO images, testing whether a model can locate the specific object in an image described by a natural-language expression (e.g., "the woman in the red hat"). Scores are typically reported as precision@1 averaged across the val, testA, and testB splits (and the related RefCOCO+ and RefCOCOg variants). LFM2.5-VL-3B averages 87.9% across these splits as of August 2026.

At a Glance

What it tests: A vision-language model's ability to ground a natural-language referring expression to the single correct object instance in an image, typically scored via bounding-box overlap or precision@1.

Why it matters: Referring expression comprehension underlies visual grounding for tool use and robotics — an agent that needs to "click the second button from the left" or "pick up the blue mug" depends on the same grounding skill RefCOCO measures.

Known limitations: RefCOCO is over a decade old and built on MS-COCO's relatively narrow object categories; frontier models now score in the high 80s–90s, making the benchmark less discriminating at the top end than newer grounding benchmarks.

What RefCOCO Measures

RefCOCO (and its variants RefCOCO+ and RefCOCOg) was introduced by Kazemzadeh et al. as a dataset of natural-language referring expressions collected via a two-player game, paired with MS-COCO images containing multiple instances of the same object category. A model must identify which specific instance the expression refers to — disambiguating between, for example, multiple people or multiple chairs in the same image.

RefCOCO+ removes location-based expressions (forcing purely appearance-based descriptions), while RefCOCOg uses longer, more descriptive expressions collected in a non-interactive setting. Scores across all splits (val, testA, testB for RefCOCO/RefCOCO+; val, test for RefCOCOg) are commonly averaged into a single "RefCOCO-avg" headline metric, as reported by Liquid AI for LFM2.5-VL-3B.

Benchmark Specifications

How RefCOCO Is Scored

For each referring expression, the model predicts a bounding box or point; the prediction is scored correct (precision@1) if it sufficiently overlaps with (or falls within) the ground-truth object's region. Per-split scores (val/testA/testB and the RefCOCO+/RefCOCOg equivalents) are commonly averaged into one headline number.

State-of-the-Art Results

Score sourced from Liquid AI's LFM2.5-VL-3B announcement, August 2026.

RefCOCO on Benchgen

No Benchgen results yet — be the first to run RefCOCO.

RefCOCO vs Other Benchmarks

Run RefCOCO on Your Model

Benchgen lets you evaluate your own vision-language model's grounding accuracy and track it across model and version changes.

Frequently Asked Questions