DocVQA — Results
| Rank | Model | Score |
|---|---|---|
| 1 | deepseek-v4-1-flash | 95.6 |
| 2 | north-micro-vision-instruct | 92.1 |
| 3 | lfm2-5-vl-3b | 91.1 |
| 4 | claude-3-5-sonnet | 0.952 |
| 5 | qwen2-5-omni-7b | 0.952 |
| 6 | qwen2-5-vl-32b-instruct | 0.948 |
| 7 | llama-4-maverick | 0.944 |
| 8 | llama-4-scout | 0.944 |
| 9 | grok-2 | 0.936 |
| 10 | nova-pro | 0.935 |
| 11 | gpt-4o | 0.928 |
| 12 | nova-lite | 0.924 |
| 13 | llama-3-2-90b-instruct | 0.901 |
| 14 | gemma-3-12b | 0.871 |
| 15 | gemma-3-27b | 0.866 |
| 16 | grok-1-5 | 0.856 |
| 17 | gemma-3-4b | 0.758 |
DocVQA
1 phaseActive
50,000-question visual QA benchmark over 12,000+ real document images — tests document layout comprehension and information retrieval from scanned pages. Metric: ANLS.
Overview
DocVQA
Quick answer: DocVQA is a visual question answering benchmark by Mathew et al. (2020) containing 50,000 questions over 12,000+ real document images. It tests AI models on document layout comprehension, information retrieval, and understanding of tables, forms, and handwriting in scanned documents. Qwen2.5 VL 72B leads with 96.4% across 26 evaluated models.
What Does DocVQA Test?
DocVQA evaluates multimodal AI on realistic document understanding tasks. Documents include forms, invoices, reports, scientific articles, and other business documents with complex layouts. Models must understand both visual structure and textual content to answer questions correctly.
| Document type | Examples |
|---|---|
| Forms | Tax forms, registration documents |
| Invoices | Business invoices, receipts |
| Scientific papers | Charts, tables, figures |
| Reports | Financial reports, company filings |
| Handwritten documents | Partially handwritten forms |
How Is DocVQA Scored?
DocVQA uses ANLS (Average Normalized Levenshtein Similarity), which measures character-level similarity between the predicted and ground-truth answer. This handles OCR imperfections and slight answer variations more robustly than exact match. Scores are reported on a 0–1 scale.
Key Facts
| Property | Value |
|---|---|
| Published | July 2020 |
| Tasks | 50,000 questions |
| Images | 12,000+ documents |
| Metric | ANLS |
| Score range | 0–1 |
| Top model | Qwen2.5 VL 72B Instruct (0.964) |
| Models evaluated | 26 |
FAQ
What is DocVQA? DocVQA is a visual question answering benchmark for document images, containing 50,000 questions over 12,000+ documents including forms, invoices, and scientific papers. It tests AI models on document layout comprehension and information retrieval.
Who created DocVQA? DocVQA was created by Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar at CVIT, IIIT Hyderabad, published in July 2020 (arXiv 2007.00398).
What metric does DocVQA use? DocVQA uses ANLS (Average Normalized Levenshtein Similarity), which tolerates minor OCR errors and formatting differences more robustly than exact match scoring.
What score does the best model achieve on DocVQA? Qwen2.5 VL 72B Instruct currently leads with 0.964 (96.4%), followed by Qwen2.5 VL 7B Instruct at 0.957 and Claude 3.5 Sonnet at 0.952.