MIMIC-CXR
1 phaseActive
377,110 de-identified chest x-rays from 227,827 imaging studies, each with 14 derived labels and a linked free-text radiology report.
Overview
MIMIC-CXR
Quick answer: MIMIC-CXR is a large, publicly available database of 377,110 de-identified chest x-rays from 227,827 imaging studies, each linked to a free-text radiology report and 14 derived labels, making it one of the most widely used foundations for benchmarking medical image classification and automated radiology report generation.
At a Glance
What it tests: Chest radiograph interpretation — including multi-label pathology classification (e.g., pneumonia, cardiomegaly, pleural effusion) and free-text radiology report generation from x-ray images.
Why it matters: A key historical bottleneck in medical computer vision was insufficient labeled data; MIMIC-CXR's scale (over 377K images) sourced from real clinical practice at Beth Israel Deaconess Medical Center made it possible to train and evaluate deep learning models for automated chest x-ray interpretation at a scale not previously available.
Known limitations: Labels are derived from two NLP tools applied to free-text reports rather than direct radiologist annotation of each image, introducing some label noise; access requires PhysioNet credentialing due to patient privacy protections, even though images are de-identified.
What MIMIC-CXR Measures
MIMIC-CXR-JPG provides a convenient, processed version of the original MIMIC-CXR database: 377,110 chest x-rays associated with 227,827 imaging studies sourced from Beth Israel Deaconess Medical Center between 2011 and 2016. Each image ships with 14 labels (e.g., no finding, cardiomegaly, edema, consolidation, pneumonia, pleural effusion) derived by applying two natural language processing labeler tools to the corresponding free-text radiology report, along with a standard reference for train/validation/test data splits.
Because every image is paired with its original free-text report, MIMIC-CXR supports two major task families: multi-label image classification (predicting which of the 14 conditions are present) and automated report generation (producing a clinically coherent free-text report directly from the image). All images have been de-identified to protect patient privacy before public release.
MIMIC-CXR's scale and public availability have made it the default training and evaluation dataset for a large fraction of published chest x-ray AI research over the past several years, and it continues to serve as a common component of broader medical multimodal benchmark suites.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Multimodal (medical image classification & radiology report generation) |
| Metric | AUROC / F1 for multi-label classification; CIDEr, BLEU, and clinical-accuracy metrics (e.g., CheXbert score) for report generation |
| Number of tasks | 377,110 chest x-ray images from 227,827 imaging studies, with 14 derived labels each |
| Saturation | Medium — classification AUROC is high on common conditions, but clinically accurate report generation remains an open challenge |
| Created by | Alistair E. W. Johnson, Tom J. Pollard, Nathaniel R. Greenbaum, Matthew P. Lungren, and collaborators |
| Source paper | Johnson et al. 2019 |
| Code | GitHub |
| Dataset | PhysioNet (credentialed access required) |
How MIMIC-CXR Is Scored
For classification tasks, models are typically scored via per-label AUROC or F1 against the 14 NLP-derived labels. For report-generation tasks, generated reports are scored with natural-language-generation metrics (BLEU, CIDEr, ROUGE) alongside clinically-oriented metrics like CheXbert or RadGraph F1, which better capture whether the generated report is clinically accurate rather than merely fluent.
State-of-the-Art Results
Scores vary substantially by task (classification vs. report generation), label subset, and evaluation protocol. See published technical reports and PhysioNet-hosted leaderboards for current standings.
MIMIC-CXR on Benchgen
No Benchgen results yet — be the first to run MIMIC-CXR.
MIMIC-CXR vs Other Benchmarks
| Benchmark | What it tests | Tasks | Saturation |
|---|---|---|---|
| MIMIC-CXR | Chest x-ray classification & report generation | 377,110 | medium |
| PMC-VQA | VQA on medical images from literature figures | 227,000 | medium |
| MedXpertQA | Expert-level medical reasoning, Text + MM | 4,460 | low |
MIMIC-CXR's real clinical chest x-ray + report pairs complement MedXpertQA's specialty-board-level clinical reasoning and PMC-VQA's literature-derived figure QA, together spanning imaging, reasoning, and literature-based medical evaluation.
Run MIMIC-CXR on Your Model
Benchgen lets teams evaluate their own medical imaging or report-generation models against MIMIC-CXR's standard splits on demand, tracking per-label classification metrics and report-generation quality over time.
Related Models & Benchmarks
Explore related multimodal benchmarks like PMC-VQA and MedXpertQA to compare imaging-focused evaluation with broader medical visual QA and expert reasoning benchmarks.