# Chartography ![Category](https://img.shields.io/badge/Category-Multimodal-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Accuracy-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) > **Quick answer:** Chartography is a visual chart recognition benchmark maintained by Surge AI that tests whether a model can read and reason over charts (bar, line, scatter, and other visualization types) directly from the image, without relying on code-execution or data-extraction tools. Claude Opus 5.5 leads the no-tools setting at 64.4% as of September 2026. ## At a Glance **What it tests:** A model's ability to extract values, trends, and relationships from a rendered chart image and answer questions about it, using vision alone rather than tool-assisted data extraction. **Why it matters:** Chart reading is a common real-world knowledge-work task (financial reports, dashboards, scientific papers), and the no-tools setting isolates genuine visual reasoning from a model's ability to call out to code or OCR tools. **Known limitations:** Chartography is run by a third party (Surge AI) rather than published as an open academic benchmark, so the task set and grading rubric are not independently inspectable; scores can also lag behind a model's latest version if the evaluator hasn't re-run it. ## What Chartography Measures Chartography evaluates a model's ability to interpret data visualizations — reading axis values, identifying trends, comparing series, and answering quantitative questions about a chart — purely from the rendered image. Surge AI reports scores both "with tools" (where a model can use code execution or other aids) and "no tools" (vision-only), which can differ substantially for the same model and should not be compared across settings. Because chart understanding sits at the intersection of vision and quantitative reasoning, labs increasingly cite Chartography alongside benchmarks like GDPval-AA and AA-Briefcase as evidence of real-world knowledge-work capability, rather than treating it as a narrow OCR test. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Multimodal (visual chart reasoning) | | Metric | % accuracy | | Settings | No-tools (vision only) and with-tools (reported separately) | | Saturation | Low | | Created by | Surge AI | | Dataset | Proprietary — not publicly released | ## How Chartography Is Scored Models answer quantitative and qualitative questions about rendered charts, graded for accuracy against ground-truth values. The "no tools" setting (used for the primary leaderboard below) disables code execution and other extraction aids, isolating vision-based chart reading. ## State-of-the-Art Results | Rank | Model | Score (no tools) | Source | Date | |---|---|---|---|---| | 1 | [Claude Opus 5.5](/models/anthropic/claude-opus-5-5) | 64.4% | [Anthropic: Claude Sonnet 5.5](https://www.anthropic.com/claude-sonnet-5-5) | 2026-09 | | 2 | [Claude Sonnet 5.5](/models/anthropic/claude-sonnet-5-5) | 61.6% | [Anthropic: Claude Sonnet 5.5](https://www.anthropic.com/claude-sonnet-5-5) | 2026-09 | | 3 | [GPT-6 Sol](/models/openai/gpt-6-sol) | 53.6% | [Anthropic: Claude Sonnet 5.5](https://www.anthropic.com/claude-sonnet-5-5) (via Surge AI) | 2026-09 | | 4 | [Claude Sonnet 5](/models/anthropic/claude-sonnet-5) | 15.6% | [Anthropic: Claude Sonnet 5.5](https://www.anthropic.com/claude-sonnet-5-5) | 2026-09 | *No-tools scores sourced from Anthropic's Claude Sonnet 5.5 announcement (September 2026), which also reports Claude Opus 5.5 and Claude Fable 5.1 "with tools" scores of 89.0% and 88.4% respectively — a separate, higher-scoring setting not directly comparable to the no-tools figures above. Anthropic notes the GPT-6 Sol figure (via Surge AI) may not yet reflect a post-launch image-understanding bugfix.* ## Chartography on Benchgen > No Benchgen results yet — [be the first to run Chartography](/benchmarks/surge-ai/chartography). ## Chartography vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **Chartography** | Visual chart reading and reasoning (no tools) | Low | | [GDPval-AA v2](/benchmarks/artificial-analysis/gdpval-aa-v2) | Real-world professional work across 44 occupations | Low | | [AA-Briefcase](/benchmarks/artificial-analysis/aa-briefcase) | Real-world knowledge-work document tasks | Low | ## Run Chartography on Your Model Benchgen lets you track multimodal chart-reasoning performance across model versions, complementing third-party-reported Chartography scores with independently repeatable evaluation. ## Related Models & Benchmarks - [Claude Opus 5.5](/models/anthropic/claude-opus-5-5) — current no-tools leader - [Claude Sonnet 5.5](/models/anthropic/claude-sonnet-5-5) - [GPT-6 Sol](/models/openai/gpt-6-sol) - [GDPval-AA v2](/benchmarks/artificial-analysis/gdpval-aa-v2) — another real-world knowledge-work benchmark these models are compared on ## Frequently Asked Questions

What is Chartography?

Chartography is a visual chart recognition benchmark run by Surge AI that tests whether a model can read and reason over data visualizations directly from the image.

What does a good score look like on Chartography?

In the no-tools setting, Claude Opus 5.5 reports the highest score at 64.4% as of September 2026, with Claude Sonnet 5.5 close behind at 61.6%. A separate "with tools" setting produces substantially higher scores (up to 89.0%) and should not be compared directly to no-tools figures.

Who created Chartography?

Chartography is created and run by Surge AI, a third-party data-labeling and evaluation company.

Why do "with tools" and "no tools" scores differ so much?

The "with tools" setting allows a model to use code execution or other extraction aids to read precise values from a chart, which tends to produce higher accuracy than relying on vision alone.

Rank	Model	Score
1	claude-opus-5-5	64.4
2	claude-sonnet-5-5	61.6
3	gpt-6-sol	53.6
4	claude-sonnet-5	15.6

Chartography — Results

Chartography — Evaluations