# MathVista ![Category](https://img.shields.io/badge/Category-Visual%20Math%20Reasoning-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Accuracy-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-6%2C141-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) ![Created](https://img.shields.io/badge/Created-Oct%202023-555) [![Paper](https://img.shields.io/badge/Paper-arXiv%202310.02255-b31b1b)](https://arxiv.org/abs/2310.02255) [![GitHub](https://img.shields.io/badge/GitHub-lupantech%2FMathVista-181717?logo=github&logoColor=white)](https://github.com/lupantech/MathVista) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/AI4Math/MathVista) > **Quick answer:** MathVista is a 6,141-problem benchmark testing whether foundation models can solve math problems grounded in charts, diagrams, plots, and other visual contexts, created by Lu et al. (2023). As of the latest results, Seed 2.1 Pro leads at 90.7% accuracy. --- ## At a Glance **What it tests:** Whether a model can correctly perceive visual evidence (charts, diagrams, geometric figures, tables) and then apply the right mathematical reasoning to it. **Why it matters:** Real-world math problems — textbooks, dashboards, scientific figures — are rarely pure text, so perception-reasoning integration is a practical requirement, not just an academic one. **Known limitations:** A 1,000-example `testmini` split is commonly used for quick evaluation, which can introduce sampling noise relative to the full 5,141-example test set. ## What MathVista Measures MathVista asks models to solve math problems grounded in charts, diagrams, plots, textbook figures, tables, and other visual contexts. It pulls from existing math and visual-question-answering datasets and adds new problems designed to require both visual perception and mathematical reasoning together — a model that only reads the topic label without parsing the chart's actual values will fail. This combination is what makes MathVista useful: many multimodal benchmarks test perception (describe the image) or reasoning (solve the equation) in isolation. MathVista forces the two together, closer to how math actually shows up in real documents and dashboards. ## Benchmark Specifications | Field | Value | |---|---| | Total problems | 6,141 | | Testmini split | 1,000 | | Test split | 5,141 | | Task category | Math / multimodal reasoning | | Metric | Accuracy | | Released | October 2023 | | Created by | Lu, Bansal, Xia, Liu, Li, Hajishirzi, Cheng, Chang, Galley, Gao (UCLA, University of Washington, Microsoft Research) | ## How MathVista Is Scored Models answer each problem (multiple-choice or free-form numeric/text response) and are graded for correctness against the ground-truth answer. The reported score is overall accuracy across the evaluated split, expressed as a percentage. ## State-of-the-Art Results ## MathVista on Benchgen > No Benchgen results yet — [be the first to run MathVista](/benchmarks/mathvista). ## MathVista vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | MathVista | Visual math reasoning across charts, diagrams, figures | 6,141 | Medium | | [MathVision](/benchmarks/mathvision) | Visual math reasoning from competition problems | 3,040 | Low | | [ChartQA](/benchmarks/chartqa) | Question answering over charts | — | — | | [DocVQA](/benchmarks/docvqa) | Question answering over document images | — | — | ## Run MathVista on Your Model Benchgen lets you run MathVista against your own model with version-controlled, regression-tracked results — useful for confirming visual math reasoning gains hold up across releases rather than relying on a single reported number. ## Related Models & Benchmarks - [Seed 2.1 Pro](/models/bytedance/seed-2-1-pro) - [MathVision](/benchmarks/mathvision) - [GPQA Diamond](/benchmarks/gpqa-diamond) ## Frequently Asked Questions

What is MathVista?

MathVista is a benchmark for evaluating mathematical reasoning of foundation models in visual contexts — 6,141 problems grounded in charts, diagrams, plots, and figures.

What does a good score look like?

The current top models score in the high 80s to low 90s (percent accuracy); scores below 75% indicate a model still struggles to combine visual perception with math reasoning reliably.

Who created MathVista?

MathVista was created by Pan Lu, Hritik Bansal, and collaborators from UCLA, the University of Washington, and Microsoft Research, published in October 2023.

Rank	Model	Score
1	lfm2-5-vl-3b	68.5
2	seed-2-1-pro	0.907
3	seed-2-1-turbo	0.905
4	o3	0.868
5	o4-mini	0.843
6	step-3-vl-10b	0.84
7	command-a-plus	0.806
8	kimi-k1-5	0.749
9	llama-4-maverick	0.737
10	gpt-4-1-mini	0.731
11	gpt-4-5	0.723

MathVista — Results

MathVista — Evaluations