LMArena Text Leaderboard

LMArena Text Leaderboard

Quick answer: The LMArena Text Leaderboard (formerly Chatbot Arena) ranks LLMs by Elo rating computed from crowdsourced, blind, pairwise human preference votes on real conversations spanning math, coding, creative writing, and other open-ended text tasks — one of the most widely cited real-world human-preference leaderboards in the field.

At a Glance

What it tests: Which model humans actually prefer in blind, head-to-head text conversations across a broad and continuously evolving mix of real-world prompt styles, rather than performance on a fixed, static test set.

Why it matters: Static benchmarks can be gamed or become contaminated over time; LMArena's live, crowdsourced, blind pairwise-comparison format captures shifting real-world human preference and remains hard to directly optimize against without genuinely improving the model.

Known limitations: Follow-up research (including "The Leaderboard Illusion," arXiv 2504.20879) has identified systematic issues such as undisclosed private testing access for some labs, meaning Elo rankings should be read alongside static benchmarks rather than as a sole source of truth.

What the LMArena Text Leaderboard Measures

LMArena (originally launched as Chatbot Arena by researchers at UC Berkeley) presents users with two anonymized model responses to the same prompt and asks them to vote for the better one, without revealing which models produced which response. Votes are aggregated using a Bradley-Terry-style statistical model to produce an Elo rating for each model — the same underlying rating system used in chess — reflecting its relative win rate against every other model in the pool. The original paper reported the platform had amassed over 240,000 votes at time of publication, a number that has grown substantially larger since, and confirmed that crowdsourced votes are sufficiently diverse and align well with expert human raters.

The Text Leaderboard specifically aggregates votes from open-ended text-to-text conversations, distinct from LMArena's separate vision, image-generation, and other modality-specific leaderboards. Because ranking is powered by live human votes rather than a fixed test set, the leaderboard continuously updates as new models are added and as user preferences shift, making it one of the most-cited real-world signals of relative LLM quality — while also being the subject of active scrutiny (including the "Leaderboard Illusion" critique) around fairness of model access and sampling during the voting period.

Benchmark Specifications

How the LMArena Text Leaderboard Is Scored

Every vote is a blind pairwise comparison: a user sees two model responses to an identical prompt (models unlabeled) and picks a winner, tie, or "both bad." These pairwise outcomes are fed into a statistical rating model (Bradley-Terry / Elo-style) that converts win rates against the full population of competing models into a single scalar rating per model, with confidence intervals derived via bootstrap resampling given the sample size of votes each model has accumulated.

State-of-the-Art Results

Elo ratings update continuously as new models are added and votes accrue. See the live LMArena Text Leaderboard for current standings.

LMArena Text Leaderboard on Benchgen

No Benchgen results yet — be the first to run LMArena Text Leaderboard.

LMArena Text Leaderboard vs Other Benchmarks

LMArena is distinguished from every static benchmark on this list by measuring live, continuously-updating human preference rather than fixed-answer correctness, making it a complementary rather than a substitute signal alongside accuracy-based benchmarks.

Run LMArena Text Leaderboard on Your Model

Benchgen tracks how your model's static benchmark performance compares against its real-world human-preference standing on LMArena, helping surface gaps between "correct" and "actually preferred by users."

Frequently Asked Questions