LogicVista

LogicVista

Quick answer: LogicVista (Xiao, Sun, Liu & Wang, 2024) is a multimodal LLM benchmark of 448 visual multiple-choice questions evaluating integrated logical reasoning — spatial, deductive, inductive, numeric, and mechanical — across diagrams, OCR, patterns, graphs, tables, 3D shapes, puzzles, sequences, and physics scenarios. Liquid AI's LFM3-VL-2B scores 60.4% as of October 2026.

At a Glance

What it tests: Five logical reasoning skills — inductive, deductive, numerical, spatial, and mechanical reasoning — applied to nine visual task formats (diagrams, OCR, patterns, graphs, tables, 3D shapes, puzzles, sequences, physics).

Why it matters: Many multimodal benchmarks focus on perception or knowledge recall; LogicVista specifically isolates logical reasoning that requires visual input, closer to the kind of structured thinking needed for navigation, puzzle-solving, and real-world planning tasks.

Known limitations: At 448 questions it is a relatively small benchmark, so per-skill breakdowns can be noisy, and each question is annotated with both a correct answer and the human-written reasoning behind it, enabling open-ended as well as multiple-choice evaluation — but most reported scores to date use the multiple-choice setting only.

What LogicVista Measures

LogicVista was built to close a gap in multimodal LLM evaluation: while MLLMs had shown fascinating emergent abilities (from writing poetry about an image to solving math problems), there was no systematic benchmark isolating their proficiency at integrated logical reasoning in visual contexts — reasoning essential for tasks like navigation and puzzle-solving.

The benchmark organizes its 448 multiple-choice questions across five reasoning-skill categories (inductive: 107, deductive: 93, numerical: 95, spatial: 79, mechanical: 74) and nine visual task-format capabilities (diagrams: 330, OCR: 234, puzzles: 256, patterns: 105, sequences: 76, tables: 70, physics: 69, graphs: 67, 3D shapes: 45 — formats overlap within questions). Each question is manually annotated with the correct answer and the reasoning behind it, so models can be evaluated in either an open-ended or strict multiple-choice mode.

Because the original paper evaluated only 8 MLLMs at release, LogicVista remains a comparatively low-saturation benchmark; scores reported since by individual labs (not yet aggregated into a public third-party leaderboard) should be read as self-reported unless independently re-run.

Benchmark Specifications

How LogicVista Is Scored

Each of the 448 multiple-choice questions is scored for correctness against the annotated answer key; the overall score is the percentage of questions answered correctly, with breakdowns available by reasoning skill (inductive/deductive/numerical/spatial/mechanical) and by visual task format.

State-of-the-Art Results

Scores above are self-reported by Liquid AI on their respective model cards/launch posts and shown for context. They are not Benchgen measurements, and are not drawn from a shared third-party leaderboard — only the 2 scores above are currently tracked locally.

LogicVista on Benchgen

No Benchgen results yet — be the first to run LogicVista.

LogicVista vs Other Benchmarks

LogicVista is narrower and more reasoning-focused than general VQA benchmarks like MMStar — use it when you specifically need to assess a model's structured logical reasoning over visual input, not general perception or knowledge recall.

Run LogicVista on Your Model

Benchgen lets you evaluate your own multimodal model against LogicVista's 448-question logical-reasoning set and track results across model versions.

Related Models & Benchmarks

• Top performers: LFM3-VL-2B · LFM2.5-VL-3B
• Related benchmarks: MMStar · BLINK

Frequently Asked Questions

Benchmark definition paraphrased from Xiao et al. 2024. State-of-the-art scores sourced from published model cards and attributed inline. Last updated 2026-10-31.