MathVista — Results
| Rank | Model | Score |
|---|---|---|
| 1 | lfm2-5-vl-3b | 68.5 |
| 2 | seed-2-1-pro | 0.907 |
| 3 | seed-2-1-turbo | 0.905 |
| 4 | o3 | 0.868 |
| 5 | o4-mini | 0.843 |
| 6 | step-3-vl-10b | 0.84 |
| 7 | command-a-plus | 0.806 |
| 8 | kimi-k1-5 | 0.749 |
| 9 | llama-4-maverick | 0.737 |
| 10 | gpt-4-1-mini | 0.731 |
| 11 | gpt-4-5 | 0.723 |
MathVista
1 phaseActive
Visual mathematical reasoning benchmark spanning charts, diagrams, and figures — 6,141 problems. Metric: accuracy. Seed 2.1 Pro reaches 90.7%.
Overview
MathVista
Quick answer: MathVista is a 6,141-problem benchmark testing whether foundation models can solve math problems grounded in charts, diagrams, plots, and other visual contexts, created by Lu et al. (2023). As of the latest results, Seed 2.1 Pro leads at 90.7% accuracy.
At a Glance
What it tests: Whether a model can correctly perceive visual evidence (charts, diagrams, geometric figures, tables) and then apply the right mathematical reasoning to it.
Why it matters: Real-world math problems — textbooks, dashboards, scientific figures — are rarely pure text, so perception-reasoning integration is a practical requirement, not just an academic one.
Known limitations: A 1,000-example testmini split is commonly used for quick evaluation, which can introduce sampling noise relative to the full 5,141-example test set.
What MathVista Measures
MathVista asks models to solve math problems grounded in charts, diagrams, plots, textbook figures, tables, and other visual contexts. It pulls from existing math and visual-question-answering datasets and adds new problems designed to require both visual perception and mathematical reasoning together — a model that only reads the topic label without parsing the chart's actual values will fail.
This combination is what makes MathVista useful: many multimodal benchmarks test perception (describe the image) or reasoning (solve the equation) in isolation. MathVista forces the two together, closer to how math actually shows up in real documents and dashboards.
Benchmark Specifications
| Field | Value |
|---|---|
| Total problems | 6,141 |
| Testmini split | 1,000 |
| Test split | 5,141 |
| Task category | Math / multimodal reasoning |
| Metric | Accuracy |
| Released | October 2023 |
| Created by | Lu, Bansal, Xia, Liu, Li, Hajishirzi, Cheng, Chang, Galley, Gao (UCLA, University of Washington, Microsoft Research) |
How MathVista Is Scored
Models answer each problem (multiple-choice or free-form numeric/text response) and are graded for correctness against the ground-truth answer. The reported score is overall accuracy across the evaluated split, expressed as a percentage.
State-of-the-Art Results
MathVista on Benchgen
No Benchgen results yet — be the first to run MathVista.
MathVista vs Other Benchmarks
| Benchmark | What it tests | Tasks | Saturation |
|---|---|---|---|
| MathVista | Visual math reasoning across charts, diagrams, figures | 6,141 | Medium |
| MathVision | Visual math reasoning from competition problems | 3,040 | Low |
| ChartQA | Question answering over charts | — | — |
| DocVQA | Question answering over document images | — | — |
Run MathVista on Your Model
Benchgen lets you run MathVista against your own model with version-controlled, regression-tracked results — useful for confirming visual math reasoning gains hold up across releases rather than relying on a single reported number.