ERQA — Results
| Rank | Model | Score |
|---|---|---|
| 1 | qwen3-8-flash-next | 72.3 |
| 2 | qwen3-8-27b | 65.5 |
ERQA
1 phaseActive
A 400-question multiple-choice benchmark for embodied reasoning — spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning — introduced alongside Gemini Robotics. Metric: % accuracy.
Overview
ERQA
Quick answer: ERQA (Embodied Reasoning Question Answering, Gemini Robotics Team, Google DeepMind, 2025) is a 400-question multiple-choice visual QA benchmark covering spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning — the embodied-reasoning skills needed for physical-world agents. Qwen3.8-Flash-Next scores 72.3% as of August 2026.
At a Glance
What it tests: Whether a multimodal model can answer embodied-reasoning visual questions — understanding spatial relationships, predicting/interpreting object and agent trajectories, reasoning about physical actions, estimating state, and integrating information across multiple camera views.
Why it matters: Introduced alongside Google DeepMind's Gemini Robotics release, ERQA isolates the embodied-reasoning capability that underlies real-world robotics and physical-agent tasks, independent of actual robot control — a clean multiple-choice diagnostic for how well a model understands physical scenes before it ever has to act on them.
Known limitations: It is a multiple-choice QA format, so it measures embodied-reasoning understanding, not closed-loop robot control performance; with only 400 questions, individual question-type breakdowns carry wider variance than larger benchmarks.
What ERQA Measures
ERQA evaluates multimodal models on 400 multiple-choice visual questions spanning five embodied-reasoning categories: spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning. It was introduced as part of Google DeepMind's "Gemini Robotics: Bringing AI into the Physical World" release to measure the embodied-reasoning foundation needed for robotics and physical-world agents, decoupled from actual robot actuation.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Reasoning (embodied/multimodal) |
| Metric | % accuracy |
| Number of questions | 400 multiple-choice |
| Saturation | Low |
| Created by | Gemini Robotics Team (Google DeepMind) |
| Source paper | Gemini Robotics Team et al. 2025 |
| GitHub | embodiedreasoning/ERQA |
How ERQA Is Scored
Models are scored on the percentage of the 400 multiple-choice embodied-reasoning questions answered correctly, covering spatial reasoning, trajectory reasoning, action reasoning, state estimation, and multi-view reasoning.
State-of-the-Art Results
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Qwen3.8-Flash-Next | 72.3% | Qwen3.8-Flash-Next model card | 2026-08 |
| 2 | Qwen3.8-27B | 65.5% | Qwen3.8-27B model card | 2026-08 |
Scores above are self-reported by Alibaba's Qwen Team on the official Hugging Face model cards and shown for context. They are not Benchgen measurements, and are not drawn from a shared third-party leaderboard — only the 2 scores above are currently tracked locally.
ERQA on Benchgen
No Benchgen results yet — be the first to run ERQA.
ERQA vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| ERQA | Embodied reasoning (spatial/trajectory/action/state/multi-view QA) | Low |
| MMBench | General vision-language reasoning across 20 ability dimensions | Medium |
| AndroidWorld | Mobile-use agent control (live device state) | Low |
ERQA is narrower and more diagnostic than general VQA benchmarks like MMBench — it specifically isolates embodied/physical-world reasoning, making it a useful companion check alongside agent-control benchmarks like AndroidWorld for models aimed at robotics or physical-agent use cases.
Run ERQA on Your Model
Benchgen lets you evaluate your own model against ERQA's 400 embodied-reasoning multiple-choice questions and track results across model versions.
Related Models & Benchmarks
- Top performers: Qwen3.8-Flash-Next · Qwen3.8-27B
- Related benchmarks: MMBench · AndroidWorld
Frequently Asked Questions
What is ERQA?
ERQA (Embodied Reasoning Question Answering) is a Google DeepMind benchmark introduced with the Gemini Robotics release (2025), testing embodied reasoning via 400 multiple-choice visual questions across spatial, trajectory, action, state-estimation, and multi-view reasoning.What does a good ERQA score look like?
Qwen3.8-Flash-Next reports 72.3% as of August 2026, the highest currently tracked on Benchgen; Qwen3.8-27B scores 65.5%.Who created ERQA?
ERQA was created by the Gemini Robotics Team at Google DeepMind; see the original paper.Is ERQA saturated?
No — ERQA remains a low-saturation benchmark; current top scores sit in the low-to-mid 70s percent, leaving substantial headroom for embodied-reasoning improvement.How does ERQA differ from AndroidWorld?
ERQA is a static multiple-choice QA benchmark testing embodied-reasoning understanding (spatial/trajectory/action/state/multi-view), while AndroidWorld tests live, closed-loop agent control of a real Android device — ERQA measures the reasoning foundation, AndroidWorld measures acting on it.Benchmark definition paraphrased from Gemini Robotics Team et al. 2025. State-of-the-art scores sourced from published model cards and attributed inline. Last updated 2026-10-31.