# CharXiv Descriptive ![Category](https://img.shields.io/badge/Category-Reasoning-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Accuracy-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) ![Created](https://img.shields.io/badge/Created-Jun%202024-555) [![Paper](https://img.shields.io/badge/Paper-arXiv%202406.18521-b31b1b)](https://arxiv.org/abs/2406.18521) [![GitHub](https://img.shields.io/badge/GitHub-princeton--nlp%2FCharXiv-181717?logo=github&logoColor=white)](https://github.com/princeton-nlp/CharXiv) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/princeton-nlp/CharXiv) > **Quick answer:** CharXiv Descriptive (`CharXiv_DQ`) is the descriptive-question sub-task of the CharXiv benchmark (Wang et al., 2024), which evaluates AI models on scientific chart understanding using figures extracted from real arXiv papers. Unlike CharXiv's Reasoning sub-task, Descriptive questions ask about directly-readable facts in a chart (axis labels, plotted values, legend entries) rather than multi-step analytical conclusions. North Micro Vision scores 60.0% as of August 2026. ## At a Glance **What it tests:** Basic, directly-verifiable chart-reading questions — reading axis labels, specific data values, legend entries, and other facts visible on a scientific chart or figure. **Why it matters:** Descriptive questions isolate whether a model can accurately perceive and extract information from a chart at all, before layering on the multi-step analytical reasoning tested by CharXiv's harder Reasoning sub-task. **Known limitations:** Because it draws from real arXiv figures rather than templated synthetic charts, chart styles and conventions vary widely, and some questions require domain-specific knowledge to interpret unusual figure types. ## What CharXiv Descriptive Measures CharXiv pairs its questions with genuine scientific figures pulled directly from arXiv papers rather than synthetically generated or templated charts, making it more representative of the messy diversity of real-world chart styles a model will encounter. The benchmark splits questions into two sub-tasks: Descriptive, covering directly-readable facts about a chart, and Reasoning, covering multi-step analytical questions that require combining several data points. The Descriptive sub-task specifically asks about facts that are directly visible in the figure — what does the x-axis represent, what is the value of a specific data point, what does a given legend entry correspond to — without requiring the model to combine multiple pieces of information or perform calculations. Because Descriptive questions test basic perception rather than compound reasoning, they act as a useful "floor" check: a model that scores much lower on Descriptive than on general VQA benchmarks likely has a genuine chart-reading weakness, rather than simply weaker multi-step reasoning. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Reasoning (chart/figure understanding) | | Metric | Accuracy | | Saturation | Medium | | Created by | Wang et al. | | Source paper | [Wang et al. 2024](https://arxiv.org/abs/2406.18521) | | GitHub | [princeton-nlp/CharXiv](https://github.com/princeton-nlp/CharXiv) | | Dataset | [princeton-nlp/CharXiv on HuggingFace](https://huggingface.co/datasets/princeton-nlp/CharXiv) | ## How CharXiv Descriptive Is Scored Each descriptive question about a chart's directly-visible facts is scored for correctness against the ground-truth answer; the overall score is the percentage of questions answered correctly on the `CharXiv_DQ` split. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [North Micro Vision Instruct](/models/cohere-labs/north-micro-vision-instruct) | 60.0% | [North Micro Vision launch blog](https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct) | 2026-08 | *Score sourced from Cohere Labs' North Micro Vision Instruct launch announcement, August 2026, evaluated on the `CharXiv_DQ` split.* ## CharXiv Descriptive on Benchgen > No Benchgen results yet — [be the first to run CharXiv Descriptive](/benchmarks/wang-et-al/charxiv-descriptive). ## CharXiv Descriptive vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **CharXiv Descriptive** | Direct fact-reading over real arXiv chart figures | Medium | | [CharXiv Reasoning](/benchmarks/wang-et-al/charxiv-reasoning) | Multi-step analytical reasoning over the same chart figures | Low | | [ChartQA](/benchmarks/vis-nlp/chartqa) | General chart QA and value extraction | Medium | Use CharXiv Descriptive alongside CharXiv Reasoning to separate basic chart-perception ability from multi-step analytical reasoning about the same underlying figures. ## Run CharXiv Descriptive on Your Model Benchgen lets you evaluate your own vision-language model's basic chart-reading ability over real scientific figures and track results across model versions. ## Related Models & Benchmarks - Top performers: [North Micro Vision Instruct](/models/cohere-labs/north-micro-vision-instruct) - Related benchmarks: [CharXiv Reasoning](/benchmarks/wang-et-al/charxiv-reasoning) · [ChartQA](/benchmarks/vis-nlp/chartqa) ## Frequently Asked Questions

What is CharXiv Descriptive?

CharXiv Descriptive is the descriptive-question sub-task of CharXiv, testing whether a model can correctly read directly-visible facts (axis labels, values, legend entries) from real scientific chart figures.

What does a good CharXiv Descriptive score look like?

North Micro Vision Instruct reports 60.0% as of August 2026; scores vary by figure complexity and vision encoder resolution.

Who created CharXiv Descriptive?

CharXiv (including its Descriptive sub-task) was created by Wang et al.; see the [original paper](https://arxiv.org/abs/2406.18521).

Is CharXiv Descriptive saturated?

It shows medium saturation — easier than CharXiv's Reasoning sub-task, but still discriminative across model sizes and vision encoder quality.

Rank	Model	Score
1	lfm3-vl-2b	87.5
2	north-micro-vision-instruct	60

CharXiv Descriptive — Results

CharXiv Descriptive — Evaluations