# EQ-Bench ![Category](https://img.shields.io/badge/Category-Reasoning-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Elo-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-60%2B-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-medium-d29922) ![Created](https://img.shields.io/badge/Created-Dec%202023-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2312.06281) [![GitHub](https://img.shields.io/badge/GitHub-EQ--bench%2FEQ--Bench-181717?logo=github&logoColor=white)](https://github.com/EQ-bench/EQ-Bench) [![Leaderboard](https://img.shields.io/badge/Leaderboard-eqbench.com-0969da)](https://eqbench.com/) > **Quick answer:** EQ-Bench is an emotional intelligence benchmark for LLMs created by Samuel Paech; it began as a 60-question quiz predicting emotional-state intensity in dialogue (correlating r=0.97 with MMLU) and has since evolved into multi-turn adversarial roleplay scenarios judged on trust-building and social-cognition skill by an LLM panel. ## At a Glance **What it tests:** A model's ability to read emotional subtext, adapt to conflicting user preferences, and navigate socially difficult conversations without collapsing into pure validation or pure confrontation. **Why it matters:** Real deployed chatbots must handle emotionally loaded, ambiguous conversations; EQ-Bench specifically stress-tests this rather than assuming helpfulness generalizes automatically from academic benchmarks. **Known limitations:** Current scoring relies entirely on LLM-judge panels with no external human-rater validation, and both the judge panel and the simulated personas may encode judge-family stylistic biases. ## What EQ-Bench Measures The original 2023 EQ-Bench asked models to predict the intensity of specific emotions felt by characters in a dialogue, a format that produced highly repeatable results from just 60 English-language questions and correlated strongly (r=0.97) with broad-capability benchmarks like MMLU — evidence that the test was capturing general intelligence-adjacent signal, not a wholly separate trait. The benchmark has since evolved substantially (current iteration: EQ-Bench 4) into a much richer format: a simulated "persona" user with hidden traits, sensitivities, and a private emotional state chats with the assessed model over 16 turns, and the transcript is scored by a panel of three LLM judges across six ability dimensions (bond & rapport, authenticity, attunement, meeting preferences & needs, emotion sensemaking, emotion management) using blind, bidirectional pairwise comparisons aggregated into an Elo rating via a Bradley–Terry fit. Roughly half the persona scenarios are adversarial by design (ego-threat navigation, power imbalance, reality distortion, social exclusion) specifically to surface failure modes that easy, cooperative scenarios would hide. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Reasoning / social-emotional intelligence | | Metric | Elo rating from LLM-judge panel (current); originally a 0-10 emotion-intensity score | | Number of tasks | 60 questions (original paper); 120 multi-turn personas (current EQ-Bench 4) | | Saturation | Medium — top models separate meaningfully on adversarial scenarios | | Created by | Samuel J. Paech | | Source paper | [Paech 2023](https://arxiv.org/abs/2312.06281) | | GitHub | [EQ-bench/EQ-Bench](https://github.com/EQ-bench/EQ-Bench) | | Leaderboard | [eqbench.com](https://eqbench.com/) | ## How EQ-Bench Is Scored In the current version, three judge models (rotated across matchups, scoring blind and bidirectionally) each pick a winner and margin per ability dimension for every head-to-head matchup; a soft Bradley–Terry fit converts these fractional pairwise outcomes into a single Elo rating per model. A separate, non-scoring "behavioural traits" pass rates each transcript 0–10 on discrete traits for informational display only. Because scores are Elo-based and relative to the current model pool, absolute numbers are less meaningful than rank position and the score gap to nearby models. ## State-of-the-Art Results > EQ-Bench's current Elo ratings shift as new judge-panel models and personas are added each release. See the [live leaderboard](https://eqbench.com/) for current standings. ## EQ-Bench on Benchgen > No Benchgen results yet — [be the first to run EQ-Bench](/benchmarks/samuel-j-paech/eq-bench). ## EQ-Bench vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **EQ-Bench** | Emotional/social intelligence in roleplay | 60-120 | medium | | [MT-Bench](/benchmarks/mt-bench) | General multi-turn conversation quality | — | high | | [WildBench](/benchmarks/bill-yuchen-lin-et-al-allen-institute-for-ai/wildbench) | Real-world query quality | 1,024 | medium | Unlike general chat-quality benchmarks, EQ-Bench specifically isolates emotional and social reasoning under adversarial persona conditions rather than measuring broad helpfulness. ## Run EQ-Bench on Your Model Benchgen lets teams run EQ-Bench-style roleplay scenarios against their own model versions, tracking Elo trends across releases to catch regressions in social-emotional handling that pure capability benchmarks wouldn't reveal. ## Frequently Asked Questions

What is EQ-Bench?

EQ-Bench is a benchmark that evaluates emotional and social intelligence in LLMs, currently through multi-turn adversarial roleplay scenarios scored by an LLM judge panel into an Elo rating.

What does a good EQ-Bench score look like?

Because the current metric is a relative Elo rating, a "good" score means ranking near the top of the live leaderboard rather than hitting a fixed absolute number.

Who created EQ-Bench?

EQ-Bench was created by independent researcher Samuel J. Paech, first published in December 2023.

EQ-Bench — Leaderboard

EQ-Bench Leaderboard