Tau2 Telecom — Results
| Rank | Model | Score |
|---|---|---|
| 1 | claude-opus-4-6 | 0.993 |
| 2 | gpt-5-4 | 0.989 |
| 3 | gpt-5-2 | 0.987 |
| 4 | claude-opus-4-5 | 0.982 |
| 5 | gpt-5-5 | 0.98 |
| 6 | claude-sonnet-4-6 | 0.979 |
| 7 | mimo-v2-pro | 0.968 |
| 8 | gpt-5 | 0.967 |
| 9 | gpt-5-1-instant | 0.956 |
| 10 | gpt-5-1-thinking | 0.956 |
| 11 | gpt-5-1 | 0.956 |
| 12 | nova-2-pro | 0.927 |
| 13 | muse-spark | 0.915 |
| 14 | minimax-m2-1 | 0.87 |
| 15 | minimax-m2 | 0.87 |
| 16 | longcat-flash-thinking | 0.831 |
| 17 | claude-haiku-4-5 | 0.83 |
| 18 | nova-2-omni | 0.8 |
| 19 | nova-2-lite | 0.76 |
| 20 | longcat-flash-chat | 0.737 |
| 21 | longcat-flash-lite | 0.728 |
| 22 | kimi-k2-instruct-0905 | 0.658 |
| 23 | kimi-k2-instruct | 0.658 |
| 24 | nemotron-3-super-120b-a12b | 0.644 |
| 25 | o3 | 0.582 |
Tau2 Telecom
1 phaseActive
τ²-Bench telecom domain — evaluates conversational agents in a dual-control environment where both agent and user use tools for shared troubleshooting. Metric: success rate.
Overview
Tau2 Telecom
Quick answer: Tau2 Telecom is the telecom domain of τ²-Bench, which evaluates conversational agents in a dual-control environment modeled as a Dec-POMDP — where both the AI agent and the simulated user can use tools — on shared telecommunications troubleshooting scenarios. LongCat-Flash-Thinking-2601 and Claude Opus 4.6 tie for the lead at 99.3% across 35 evaluated models.
What Does Tau2 Telecom Test?
Unlike the original τ-bench, where only the agent uses tools, τ²-Bench introduces a dual-control setting: both the agent and the (simulated) user can invoke tools to resolve a shared telecom troubleshooting task, such as diagnosing a connectivity issue that requires actions on both sides. This tests coordination, communication, and joint problem-solving rather than just one-sided tool execution.
| Focus area | Examples |
|---|---|
| Dual-control coordination | Agent and user both act on a shared environment via tools |
| Communication | Clearly conveying instructions the user must execute themselves |
| Troubleshooting | Diagnosing and resolving telecom connectivity/service issues |
| Tool calling | Invoking diagnostic and account-management APIs |
How Is Tau2 Telecom Scored?
Each task is graded by comparing the final environment state (across both agent-controlled and user-controlled actions) to a ground-truth expected outcome. A task is scored as fully successful (1) or failed (0); the reported score is the average success rate (pass^1) across the task suite, normalized to 0–1.
Key Facts
| Property | Value |
|---|---|
| Released | June 2025 |
| Created by | Sierra Research |
| Domain | Telecom troubleshooting (dual-control) |
| Metric | Success rate |
| Score range | 0–1 |
| Top model | LongCat-Flash-Thinking-2601 / Claude Opus 4.6 (0.993) |
| Models evaluated | 35 |
FAQ
What is Tau2 Telecom? Tau2 Telecom is the telecom domain of τ²-Bench, a benchmark evaluating conversational AI agents in a dual-control environment where both the agent and a simulated user use tools to jointly resolve telecommunications troubleshooting tasks.
Who created Tau2 Telecom? τ²-Bench was created by Sierra Research — Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and collaborators — published in June 2025, as a successor to the original τ-bench.
How is τ²-Bench different from τ-bench (TAU-bench)? The original τ-bench only lets the agent use tools while the user is a passive information provider. τ²-Bench models the interaction as a dual-control Dec-POMDP, where the user must also take tool-based actions — a more realistic simulation of scenarios like telecom troubleshooting where the customer performs steps on their own device.
What score does the best model achieve on Tau2 Telecom? LongCat-Flash-Thinking-2601 (Meituan) and Claude Opus 4.6 (Anthropic) tie for the lead at 0.993 (99.3%), with GPT-5.4 close behind at 0.989.
Is Tau2 Telecom saturated? Near the top, yes — the leading models cluster above 0.95. But scores drop off sharply further down the leaderboard, with several models below 0.5 and the weakest at 0.132, so the benchmark still separates strong from weak agentic tool-use.