# LMArena Text Leaderboard ![Category](https://img.shields.io/badge/Category-Preference-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Elo-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-240K%2B%20votes-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-medium-d29922) ![Created](https://img.shields.io/badge/Created-Mar%202024-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2403.04132) [![Leaderboard](https://img.shields.io/badge/Leaderboard-lmarena.ai-0969da)](https://lmarena.ai/leaderboard) > **Quick answer:** The LMArena Text Leaderboard (formerly Chatbot Arena) ranks LLMs by Elo rating computed from crowdsourced, blind, pairwise human preference votes on real conversations spanning math, coding, creative writing, and other open-ended text tasks — one of the most widely cited real-world human-preference leaderboards in the field. ## At a Glance **What it tests:** Which model humans actually prefer in blind, head-to-head text conversations across a broad and continuously evolving mix of real-world prompt styles, rather than performance on a fixed, static test set. **Why it matters:** Static benchmarks can be gamed or become contaminated over time; LMArena's live, crowdsourced, blind pairwise-comparison format captures shifting real-world human preference and remains hard to directly optimize against without genuinely improving the model. **Known limitations:** Follow-up research (including "The Leaderboard Illusion," arXiv 2504.20879) has identified systematic issues such as undisclosed private testing access for some labs, meaning Elo rankings should be read alongside static benchmarks rather than as a sole source of truth. ## What the LMArena Text Leaderboard Measures LMArena (originally launched as Chatbot Arena by researchers at UC Berkeley) presents users with two anonymized model responses to the same prompt and asks them to vote for the better one, without revealing which models produced which response. Votes are aggregated using a Bradley-Terry-style statistical model to produce an Elo rating for each model — the same underlying rating system used in chess — reflecting its relative win rate against every other model in the pool. The original paper reported the platform had amassed over 240,000 votes at time of publication, a number that has grown substantially larger since, and confirmed that crowdsourced votes are sufficiently diverse and align well with expert human raters. The Text Leaderboard specifically aggregates votes from open-ended text-to-text conversations, distinct from LMArena's separate vision, image-generation, and other modality-specific leaderboards. Because ranking is powered by live human votes rather than a fixed test set, the leaderboard continuously updates as new models are added and as user preferences shift, making it one of the most-cited real-world signals of relative LLM quality — while also being the subject of active scrutiny (including the "Leaderboard Illusion" critique) around fairness of model access and sampling during the voting period. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Preference / crowdsourced pairwise human evaluation | | Metric | Elo rating (Bradley-Terry model over pairwise win/loss votes) | | Number of tasks | 240,000+ votes at original publication; continuously growing live | | Saturation | Medium — top models cluster closely, with rankings shifting release to release | | Created by | Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, et al. (LMArena, UC Berkeley) | | Source paper | [Chiang et al. 2024](https://arxiv.org/abs/2403.04132) | | Live leaderboard | [lmarena.ai/leaderboard](https://lmarena.ai/leaderboard) | ## How the LMArena Text Leaderboard Is Scored Every vote is a blind pairwise comparison: a user sees two model responses to an identical prompt (models unlabeled) and picks a winner, tie, or "both bad." These pairwise outcomes are fed into a statistical rating model (Bradley-Terry / Elo-style) that converts win rates against the full population of competing models into a single scalar rating per model, with confidence intervals derived via bootstrap resampling given the sample size of votes each model has accumulated. ## State-of-the-Art Results > Elo ratings update continuously as new models are added and votes accrue. See the [live LMArena Text Leaderboard](https://lmarena.ai/leaderboard) for current standings. ## LMArena Text Leaderboard on Benchgen > No Benchgen results yet — [be the first to run LMArena Text Leaderboard](/benchmarks/wei-lin-chiang-lianmin-zheng-ying-sheng-anastasios-nikolas-angelopoulos-et-al-lmarena-uc-berkeley/lmarena-text-leaderboard). ## LMArena Text Leaderboard vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **LMArena Text Leaderboard** | Crowdsourced human pairwise preference (text) | 240K+ votes | medium | | [MASK](/benchmarks/richard-ren-arunim-agarwal-mantas-mazeika-et-al-center-for-ai-safety/mask) | Honesty under pressure | Large-scale | low | | [GPQA](/benchmarks/gpqa) | Graduate-level science question answering | — | low | LMArena is distinguished from every static benchmark on this list by measuring live, continuously-updating human preference rather than fixed-answer correctness, making it a complementary rather than a substitute signal alongside accuracy-based benchmarks. ## Run LMArena Text Leaderboard on Your Model Benchgen tracks how your model's static benchmark performance compares against its real-world human-preference standing on LMArena, helping surface gaps between "correct" and "actually preferred by users." ## Frequently Asked Questions

What is the LMArena Text Leaderboard?

The LMArena Text Leaderboard (formerly Chatbot Arena) ranks LLMs by Elo rating computed from crowdsourced blind pairwise human preference votes on real text conversations.

What does a good LMArena score look like?

Elo ratings are relative, not absolute — a "good" score means ranking near the top of the current live leaderboard, since ratings continuously shift as new models are added and preferences change.

Who created the LMArena Text Leaderboard?

LMArena was created by Wei-Lin Chiang, Lianmin Zheng, and collaborators, originally launched from UC Berkeley as Chatbot Arena, with the founding paper published in March 2024.

LMArena Text Leaderboard — Leaderboard

LMArena Text Leaderboard Leaderboard