OmniDocBench — Results
| Rank | Model | Score |
|---|---|---|
| 1 | kimi-k3 | 91.1 |
| 2 | muse-glimmer | 75.8 |
OmniDocBench
1 phaseActive
Diverse PDF document-parsing benchmark spanning academic papers, textbooks, handwritten notes, and newspapers. CVPR 2025. Metric: % accuracy / parsing quality.
Overview
OmniDocBench
Quick answer: OmniDocBench (Ouyang et al., CVPR 2025) is a benchmark for diverse PDF document parsing, featuring high-quality annotations across 9 document sources — academic papers, textbooks, handwritten notes, densely typeset newspapers, and more — with 19 layout categories and 15 attribute labels supporting both end-to-end and task-specific evaluation. Kimi K3 scores 91.1% as of July 2026.
At a Glance
What it tests: A model's ability to accurately parse and extract structured content (text, tables, formulas, layout) from diverse real-world PDF documents, including challenging cases like handwritten notes and dense newspaper layouts.
Why it matters: Document parsing underpins LLM data pipelines and retrieval-augmented generation systems. OmniDocBench's breadth across document types and fine-grained, attribute-level evaluation makes it a more realistic and rigorous test than prior narrow-domain document benchmarks.
Known limitations: As a vision-heavy benchmark, results depend on how well a model's vision encoder handles small-font text, complex tables, and handwriting — text-only models cannot be evaluated.
What OmniDocBench Measures
OmniDocBench evaluates document content extraction across nine document sources, including academic papers, textbooks, and more challenging cases such as handwritten notes and densely typeset newspapers. It supports flexible, multi-level evaluations — from end-to-end document parsing to task-specific and attribute-based analysis using 19 layout categories (e.g., tables, formulas, headers) and 15 attribute labels (e.g., font size, rotation, language). This structure allows fair comparison of both pipeline-based parsing methods and end-to-end vision-language models.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Document parsing / OCR reasoning |
| Metric | % accuracy (parsing quality) |
| Document sources | 9 types |
| Layout categories | 19 |
| Attribute labels | 15 |
| Saturation | Low |
| Created by | Ouyang et al. |
| Source paper | OmniDocBench (arXiv 2412.07626, CVPR 2025) |
| GitHub | opendatalab/OmniDocBench |
How OmniDocBench Is Scored
Models parse each document and are scored against ground-truth annotations across layout and attribute dimensions, producing both an aggregate accuracy score and fine-grained breakdowns by document type and content category (text, tables, formulas, reading order).
State-of-the-Art Results
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Kimi K3 | 91.1% | Kimi K3 technical report | 2026-07 |
Score sourced from Moonshot AI's Kimi K3 announcement, July 2026.
OmniDocBench on Benchgen
No Benchgen results yet — be the first to run OmniDocBench.
OmniDocBench vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| OmniDocBench | Diverse PDF document parsing | Low |
| CharXiv Reasoning | Scientific chart/figure reasoning | Low |
| MMMU-Pro | Multi-discipline multimodal understanding | Low |
| OfficeQA Pro | Office document question-answering | Low |
Run OmniDocBench on Your Model
Benchgen lets you run OmniDocBench against your own multimodal model, tracking document parsing accuracy across content types and layout categories to catch regressions in production document pipelines.