# WideSearch ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Task%20Success-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-200-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-Aug%202025-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2508.07999) [![GitHub](https://img.shields.io/badge/GitHub-ByteDance--Seed%2FWideSearch-181717?logo=github&logoColor=white)](https://github.com/ByteDance-Seed/WideSearch) > **Quick answer:** WideSearch is a benchmark of 200 manually curated questions (100 English, 100 Chinese) across 15+ domains that tests whether agentic search systems can reliably collect large-scale, atomic pieces of information from the web and organize them into a complete, verifiable output — a task on which most current systems succeed less than 5% of the time. ## At a Glance **What it tests:** Whether single-agent, multi-agent, and commercial agentic search systems can perform "wide-context" information collection — gathering many individually-verifiable facts (not just one deep answer) and assembling them correctly. **Why it matters:** Many real workflows (market research, competitive analysis, trip planning) require broad, exhaustive, and accurate information-gathering rather than a single deep answer; WideSearch shows that even strong commercial search agents currently fail almost all of these tasks. **Known limitations:** At 200 tasks, WideSearch is modest in scale, and because task difficulty is calibrated so that human testers can achieve near-100% accuracy with sufficient time, gains on WideSearch mainly reflect agent reliability rather than raw task difficulty for humans. ## What WideSearch Measures WideSearch's 200 questions are grounded in real user queries spanning over 15 diverse domains, and were passed through a rigorous five-stage quality-control pipeline to ensure each task's difficulty, completeness, and verifiability. Each task requires an agent to collect large-scale atomic information — individually verifiable facts — and arrange it into a well-organized final output, mirroring real-world workflows like a financial analyst compiling metrics across every company in a sector. The paper benchmarks more than 10 state-of-the-art agentic search systems, spanning single-agent setups, multi-agent frameworks, and end-to-end commercial products. Most systems achieve overall success rates near 0%, with the best performer reaching only 5% — while cross-validated human testers, given sufficient time, can achieve close to 100% success. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent / broad, verifiable information-seeking | | Metric | Task success rate (%), graded against per-item verifiable ground truth | | Number of tasks | 200 questions (100 English, 100 Chinese) across 15+ domains | | Saturation | Very low — best system reaches only 5% success versus near-100% for humans | | Created by | Ryan Wong, Jiawei Wang, Junjie Zhao, Li Chen, Yan Gao, Long Zhang, Xuan Zhou, Zuo Wang, Kai Xiang, et al. (ByteDance Seed) | | Source paper | [Wong et al. 2025](https://arxiv.org/abs/2508.07999) | | GitHub | [ByteDance-Seed/WideSearch](https://github.com/ByteDance-Seed/WideSearch) | ## How WideSearch Is Scored Each task's expected output is a set of atomic, individually-verifiable facts arranged in a structured format; an agent's response is graded by checking each atomic fact against ground truth, and overall success rate reflects the share of tasks where the agent's compiled output is judged complete and correct. ## State-of-the-Art Results > The WideSearch paper reports that most of the more than 10 evaluated agentic search systems — including single-agent, multi-agent, and commercial end-to-end products — achieve overall success rates near 0%, with the best performer reaching just 5%. See the [project page](https://widesearch-seed.github.io/) for the full per-system breakdown. ## WideSearch on Benchgen > No Benchgen results yet — [be the first to run WideSearch](/benchmarks/ryan-wong-jiawei-wang-junjie-zhao-li-chen-yan-gao-long-zhang-xuan-zhou-zuo-wang-kai-xiang-et-al-bytedance-seed/widesearch). ## WideSearch vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **WideSearch** | Broad, verifiable web information-gathering | 200 | very low | | [FRAMES](/benchmarks/frames) | Multi-hop factual reasoning and retrieval | — | — | | [Vending-Bench 2](/benchmarks/andon-labs/vending-bench-2) | Long-horizon agent coherence via simulated business | 1 (year-long run) | low | WideSearch is distinguished from single-answer retrieval benchmarks by requiring exhaustive, breadth-first collection of many verifiable facts rather than one deep or narrow answer, exposing a distinct failure mode current search agents largely have not solved. ## Run WideSearch on Your Model Benchgen lets teams evaluate their own agent against WideSearch's broad information-gathering tasks, surfacing whether it can reliably compile complete, verifiable results rather than partial or hallucinated ones. ## Frequently Asked Questions

What is WideSearch?

WideSearch is a 200-question benchmark that tests whether agentic search systems can reliably collect and organize large-scale, individually-verifiable information from the web.

What does a good WideSearch score look like?

Most current agentic search systems score near 0% on WideSearch, with the best performer reaching only 5%, so any meaningfully non-zero, consistent success rate represents a strong result relative to the current state of the art.

Who created WideSearch?

WideSearch was created by Ryan Wong and collaborators at ByteDance Seed, published in August 2025.

WideSearch — Leaderboard

WideSearch Leaderboard