TAU-bench Retail — Results
| Rank | Model | Score |
|---|---|---|
| 1 | claude-sonnet-4-5 | 0.862 |
| 2 | claude-opus-4-1 | 0.824 |
| 3 | claude-opus-4 | 0.814 |
| 4 | claude-sonnet-4 | 0.805 |
| 5 | glm-4-5 | 0.797 |
| 6 | glm-4-5-air | 0.779 |
| 7 | qwen3-coder-480b-a35b | 0.775 |
| 8 | o4-mini | 0.718 |
| 9 | o1 | 0.708 |
| 10 | qwen3-next-80b-a3b-thinking | 0.696 |
| 11 | claude-3-5-sonnet | 0.692 |
| 12 | gpt-4-5 | 0.684 |
| 13 | gpt-4-1 | 0.68 |
| 14 | minimax-m1-40k | 0.678 |
| 15 | qwen3-235b-a22b-thinking-2507 | 0.678 |
| 16 | minimax-m1-80k | 0.635 |
| 17 | qwen3-next-80b-a3b-instruct | 0.609 |
| 18 | gpt-4o | 0.603 |
| 19 | o3-mini | 0.576 |
| 20 | gpt-4-1-mini | 0.558 |
| 21 | claude-3-5-haiku | 0.51 |
TAU-bench Retail
1 phaseActive
Evaluates language agents on retail domain conversations — order cancellations, address changes, and status checks via API tools and policy rules. Metric: success rate.
Overview
TAU-bench Retail
Quick answer: TAU-bench Retail is the retail domain of τ-bench (TAU-bench), evaluating language agents' ability to handle dynamic, multi-turn conversations with simulated users while using domain-specific API tools and following policy guidelines — tasks like order cancellations, address changes, and order status checks. Claude Sonnet 4.5 leads with 86.2% across 25 evaluated models.
What Does TAU-bench Retail Test?
TAU-bench Retail evaluates tool-agent-user interaction in a retail customer-service setting. Unlike static tool-calling benchmarks, agents must maintain coherent multi-turn conversations with a simulated user, correctly interpret ambiguous requests, invoke the right API tools, and strictly follow retail-specific policy rules (e.g., refund eligibility, order modification windows).
| Focus area | Examples |
|---|---|
| Multi-turn dialogue | Sustaining context across a realistic customer service conversation |
| Tool calling | Invoking retail APIs (order lookup, cancellation, address update) |
| Policy adherence | Following domain rules for refunds, exchanges, and account changes |
| User simulation | Agent is evaluated against an LLM-simulated user with hidden intents |
How Is TAU-bench Retail Scored?
Each task is graded by comparing the final database/environment state after the conversation to a ground-truth expected state, plus checking that any required communications to the user occurred. A task is scored as fully successful (1) or failed (0) based on strict outcome matching; the reported score is the average success rate (pass^1) across the task suite, normalized to 0–1.
Key Facts
| Property | Value |
|---|---|
| Released | June 2024 |
| Created by | Sierra Research |
| Domain | Retail customer service |
| Metric | Success rate |
| Score range | 0–1 |
| Top model | Claude Sonnet 4.5 (0.862) |
| Models evaluated | 25 |
FAQ
What is TAU-bench Retail? TAU-bench Retail is the retail domain of τ-bench, a benchmark for tool-agent-user interaction that evaluates language agents on realistic customer-service conversations requiring tool use and policy compliance.
Who created TAU-bench Retail? TAU-bench (τ-bench) was created by Sierra Research — Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan — published in June 2024.
How is TAU-bench Retail different from standard tool-calling benchmarks? It requires sustained multi-turn dialogue with a simulated user rather than single-shot tool calls, and grades agents on final environment state plus policy compliance rather than just correct API syntax.
What score does the best model achieve on TAU-bench Retail? Claude Sonnet 4.5 leads with 0.862 (86.2%), followed by Claude Opus 4.1 at 0.824 and Claude Opus 4 at 0.814. GLM-4.5 is the top open-weight model at 0.797.
Is TAU-bench Retail saturated? No. The average score across all evaluated models is around 0.7, with a wide spread from 0.226 to 0.862 — the benchmark still meaningfully separates frontier from weaker models.