HealthBench — Results
| Rank | Model | Score |
|---|---|---|
| 1 | kimi-k2-thinking-0905 | 0.58 |
| 2 | gpt-oss-120b-high | 0.576 |
| 3 | gpt-5-6-sol | 0.57 |
| 4 | gpt-5-6-terra | 0.57 |
| 5 | gpt-5-6-luna | 0.558 |
| 6 | gpt-5-3-codex | 0.541 |
| 7 | gpt-5-5 | 0.514 |
HealthBench
1 phaseActive
5,000-conversation healthcare benchmark graded by 262 physicians — measures AI performance and safety across health contexts and behavioral dimensions. Metric: rubric score.
Overview
HealthBench
Quick answer: HealthBench is an open-source healthcare benchmark by Arora et al. (2025) at OpenAI, consisting of 5,000 multi-turn conversations graded by 262 physicians using 48,562 unique rubric criteria. It measures both performance and safety of AI models in health contexts. Qwen3.8 Max leads with 60.2% across 9 evaluated models.
What Does HealthBench Test?
HealthBench evaluates AI models on realistic multi-turn medical conversations covering diverse health contexts and behavioral dimensions. The benchmark is physician-graded — each conversation is scored against rubric criteria written by 262 medical doctors, making it one of the most expert-validated healthcare benchmarks available.
| Dimension | What it tests |
|---|---|
| Medical accuracy | Correctness of health information provided |
| Safety | Avoidance of harmful advice |
| Communication | Clarity and appropriateness for patients |
| Clinician use cases | Professional-grade responses for practitioners |
HealthBench also has two sub-benchmarks: HealthBench Consensus (highest physician agreement questions) and HealthBench Professional (clinician use cases).
How Is HealthBench Scored?
Each conversation is scored against physician-authored rubric criteria. Scores represent the fraction of rubric criteria met, normalized to 0–1. The overall score reflects both helpfulness and safety across the full conversation.
Key Facts
| Property | Value |
|---|---|
| Published | May 2025 |
| Conversations | 5,000 |
| Physician graders | 262 |
| Rubric criteria | 48,562 |
| Metric | Rubric score |
| Score range | 0–1 |
| Top model | Qwen3.8 Max (0.602) |
| Models evaluated | 9 |
FAQ
What is HealthBench? HealthBench is an open-source benchmark for measuring the performance and safety of large language models in healthcare, consisting of 5,000 multi-turn conversations evaluated by 262 physicians using 48,562 unique rubric criteria.
Who created HealthBench? HealthBench was created by Rahul K. Arora, Jason Wei, and colleagues at OpenAI, published in May 2025 (arXiv 2505.08775).
What are the HealthBench sub-benchmarks? HealthBench Consensus focuses on questions with especially high physician agreement. HealthBench Professional evaluates model capability for clinician use cases using real clinician-style chats.
What score does the best model achieve on HealthBench? Qwen3.8 Max from Alibaba Cloud currently leads with 0.602 (60.2%), followed by Kimi K2-Thinking-0905 at 0.580 and GPT OSS 120B at 0.576.