CharXiv Descriptive — Results
| Rank | Model | Score |
|---|---|---|
| 1 | north-micro-vision-instruct | 60 |
CharXiv Descriptive
1 phaseActive
Descriptive-question sub-task of CharXiv — basic chart-reading questions over real scientific figures extracted from arXiv papers.
Overview
CharXiv Descriptive
Quick answer: CharXiv Descriptive (
CharXiv_DQ) is the descriptive-question sub-task of the CharXiv benchmark (Wang et al., 2024), which evaluates AI models on scientific chart understanding using figures extracted from real arXiv papers. Unlike CharXiv's Reasoning sub-task, Descriptive questions ask about directly-readable facts in a chart (axis labels, plotted values, legend entries) rather than multi-step analytical conclusions. North Micro Vision scores 60.0% as of August 2026.
At a Glance
What it tests: Basic, directly-verifiable chart-reading questions — reading axis labels, specific data values, legend entries, and other facts visible on a scientific chart or figure.
Why it matters: Descriptive questions isolate whether a model can accurately perceive and extract information from a chart at all, before layering on the multi-step analytical reasoning tested by CharXiv's harder Reasoning sub-task.
Known limitations: Because it draws from real arXiv figures rather than templated synthetic charts, chart styles and conventions vary widely, and some questions require domain-specific knowledge to interpret unusual figure types.
What CharXiv Descriptive Measures
CharXiv pairs its questions with genuine scientific figures pulled directly from arXiv papers rather than synthetically generated or templated charts, making it more representative of the messy diversity of real-world chart styles a model will encounter. The benchmark splits questions into two sub-tasks: Descriptive, covering directly-readable facts about a chart, and Reasoning, covering multi-step analytical questions that require combining several data points.
The Descriptive sub-task specifically asks about facts that are directly visible in the figure — what does the x-axis represent, what is the value of a specific data point, what does a given legend entry correspond to — without requiring the model to combine multiple pieces of information or perform calculations.
Because Descriptive questions test basic perception rather than compound reasoning, they act as a useful "floor" check: a model that scores much lower on Descriptive than on general VQA benchmarks likely has a genuine chart-reading weakness, rather than simply weaker multi-step reasoning.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Reasoning (chart/figure understanding) |
| Metric | Accuracy |
| Saturation | Medium |
| Created by | Wang et al. |
| Source paper | Wang et al. 2024 |
| GitHub | princeton-nlp/CharXiv |
| Dataset | princeton-nlp/CharXiv on HuggingFace |
How CharXiv Descriptive Is Scored
Each descriptive question about a chart's directly-visible facts is scored for correctness against the ground-truth answer; the overall score is the percentage of questions answered correctly on the CharXiv_DQ split.
State-of-the-Art Results
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | North Micro Vision Instruct | 60.0% | North Micro Vision launch blog | 2026-08 |
Score sourced from Cohere Labs' North Micro Vision Instruct launch announcement, August 2026, evaluated on the CharXiv_DQ split.
CharXiv Descriptive on Benchgen
No Benchgen results yet — be the first to run CharXiv Descriptive.
CharXiv Descriptive vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| CharXiv Descriptive | Direct fact-reading over real arXiv chart figures | Medium |
| CharXiv Reasoning | Multi-step analytical reasoning over the same chart figures | Low |
| ChartQA | General chart QA and value extraction | Medium |
Use CharXiv Descriptive alongside CharXiv Reasoning to separate basic chart-perception ability from multi-step analytical reasoning about the same underlying figures.
Run CharXiv Descriptive on Your Model
Benchgen lets you evaluate your own vision-language model's basic chart-reading ability over real scientific figures and track results across model versions.
Related Models & Benchmarks
- Top performers: North Micro Vision Instruct
- Related benchmarks: CharXiv Reasoning · ChartQA