# OmniDocBench ![Category](https://img.shields.io/badge/Category-Document%20Parsing-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Accuracy-2da44e) ![Sources](https://img.shields.io/badge/Sources-9%20document%20types-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-Dec%202024-555) [![Paper](https://img.shields.io/badge/Paper-arXiv%202412.07626-b31b1b)](https://arxiv.org/abs/2412.07626) [![GitHub](https://img.shields.io/badge/GitHub-opendatalab%2FOmniDocBench-181717?logo=github&logoColor=white)](https://github.com/opendatalab/OmniDocBench) > **Quick answer:** OmniDocBench (Ouyang et al., CVPR 2025) is a benchmark for diverse PDF document parsing, featuring high-quality annotations across 9 document sources — academic papers, textbooks, handwritten notes, densely typeset newspapers, and more — with 19 layout categories and 15 attribute labels supporting both end-to-end and task-specific evaluation. Kimi K3 scores 91.1% as of July 2026. ## At a Glance **What it tests:** A model's ability to accurately parse and extract structured content (text, tables, formulas, layout) from diverse real-world PDF documents, including challenging cases like handwritten notes and dense newspaper layouts. **Why it matters:** Document parsing underpins LLM data pipelines and retrieval-augmented generation systems. OmniDocBench's breadth across document types and fine-grained, attribute-level evaluation makes it a more realistic and rigorous test than prior narrow-domain document benchmarks. **Known limitations:** As a vision-heavy benchmark, results depend on how well a model's vision encoder handles small-font text, complex tables, and handwriting — text-only models cannot be evaluated. ## What OmniDocBench Measures OmniDocBench evaluates document content extraction across nine document sources, including academic papers, textbooks, and more challenging cases such as handwritten notes and densely typeset newspapers. It supports flexible, multi-level evaluations — from end-to-end document parsing to task-specific and attribute-based analysis using 19 layout categories (e.g., tables, formulas, headers) and 15 attribute labels (e.g., font size, rotation, language). This structure allows fair comparison of both pipeline-based parsing methods and end-to-end vision-language models. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Document parsing / OCR reasoning | | Metric | % accuracy (parsing quality) | | Document sources | 9 types | | Layout categories | 19 | | Attribute labels | 15 | | Saturation | Low | | Created by | Ouyang et al. | | Source paper | [OmniDocBench (arXiv 2412.07626, CVPR 2025)](https://arxiv.org/abs/2412.07626) | | GitHub | [opendatalab/OmniDocBench](https://github.com/opendatalab/OmniDocBench) | ## How OmniDocBench Is Scored Models parse each document and are scored against ground-truth annotations across layout and attribute dimensions, producing both an aggregate accuracy score and fine-grained breakdowns by document type and content category (text, tables, formulas, reading order). ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Kimi K3](/models/moonshot-ai/kimi-k3) | 91.1% | [Kimi K3 technical report](https://www.kimi.com/blog/kimi-k3) | 2026-07 | *Score sourced from Moonshot AI's Kimi K3 announcement, July 2026.* ## OmniDocBench on Benchgen > No Benchgen results yet — [be the first to run OmniDocBench](/benchmarks/opendatalab/omnidocbench). ## OmniDocBench vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **OmniDocBench** | Diverse PDF document parsing | Low | | [CharXiv Reasoning](/benchmarks/wang-et-al/charxiv-reasoning) | Scientific chart/figure reasoning | Low | | [MMMU-Pro](/benchmarks/tiger-ai-lab/mmmu-pro) | Multi-discipline multimodal understanding | Low | | [OfficeQA Pro](/benchmarks/databricks/officeqa-pro) | Office document question-answering | Low | ## Run OmniDocBench on Your Model Benchgen lets you run OmniDocBench against your own multimodal model, tracking document parsing accuracy across content types and layout categories to catch regressions in production document pipelines. ## Frequently Asked Questions

What is OmniDocBench?

OmniDocBench is a benchmark for diverse PDF document parsing, evaluating models across 9 document sources (academic papers, textbooks, handwritten notes, newspapers, and more) with fine-grained layout and attribute annotations.

What does a good score look like on OmniDocBench?

Scores above 90% indicate strong, production-grade document parsing across most document types. Kimi K3 reports 91.1% as of July 2026.

Who created OmniDocBench?

OmniDocBench was created by Linke Ouyang and collaborators, accepted at CVPR 2025 (arXiv:2412.07626). Code and data are maintained by OpenDataLab.

Rank	Model	Score
1	nex-n2-5-pro	92.2
2	sage-a1-5	91.5
3	kimi-k3	91.1
4	qwen3-8-27b	91.1
5	muse-glimmer	75.8

OmniDocBench — Results

OmniDocBench — Evaluations