# WebArena-Verified ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Task%20Success-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-812-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2023%2F2025-555) [![Paper](https://img.shields.io/badge/Paper-arXiv%202307.13854-b31b1b)](https://arxiv.org/abs/2307.13854) [![GitHub](https://img.shields.io/badge/GitHub-ServiceNow%2Fwebarena--verified-181717?logo=github&logoColor=white)](https://github.com/ServiceNow/webarena-verified) > **Quick answer:** WebArena-Verified is an audited, deterministically-scored re-release (ServiceNow, 2025) of the original WebArena benchmark (Zhou et al., CMU, 2023) — a fully functional, self-hostable web environment with 812 realistic long-horizon tasks across e-commerce, social-forum, GitLab, and content-management sites, used to test autonomous browser-use agents. Qwen3.8-27B scores 64.8% as of August 2026. ## At a Glance **What it tests:** Whether an autonomous agent can complete realistic, long-horizon natural-language tasks inside a fully functional web environment spanning four site categories (e-commerce, social forum discussions, collaborative software development/GitLab, content management), with success verified deterministically rather than by fuzzy string-matching or LLM-as-judge. **Why it matters:** The original 2023 WebArena paper found even strong GPT-4-based agents succeeded on only 14.4% of tasks (vs. 78.2% human performance), exposing a large gap between synthetic agent benchmarks and realistic browser use. WebArena-Verified audits every task, reference answer, and evaluator from that original release and replaces nondeterministic judging with type-aware, structural comparison, making scores more reliable and reproducible across agents and model versions. **Known limitations:** The underlying task set is the same 812 tasks as the original 2023 WebArena (re-audited, not newly authored), so it inherits that benchmark's site/domain scope; a curated 258-task "Hard" subset exists for faster, lower-cost evaluation but is a different, smaller scope than the full-dataset score reported here. ## What WebArena-Verified Measures WebArena-Verified distributes a curated, version-controlled dataset of the original WebArena's 812 web tasks together with deterministic evaluators that operate on agent responses and captured network traces, enabling offline evaluation without a live web environment. It covers four realistic, self-hosted site categories — e-commerce, social forum discussions, GitLab-style collaborative development, and content management — enriched with tools (e.g. a map) and external knowledge bases (e.g. user manuals) to encourage human-like task-solving, exactly as specified in the original WebArena environment. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent (browser-use) | | Metric | % task success | | Number of tasks | 812 (full verified dataset; a 258-task "Hard" subset is also available) | | Saturation | Low | | Created by | Zhou et al. (original WebArena, CMU); El hattami, Thakkar, Chapados, Pal (Verified re-release, ServiceNow) | | Source paper | [Zhou et al. 2023](https://arxiv.org/abs/2307.13854) | | GitHub | [ServiceNow/webarena-verified](https://github.com/ServiceNow/webarena-verified) | ## How WebArena-Verified Is Scored Each of the 812 tasks is scored by a deterministic, type-aware evaluator that compares an agent's response and captured network trace against a manually-reviewed reference answer — replacing the original WebArena's LLM-as-judge and substring-matching scoring with structural comparison; the overall score is the percentage of tasks completed successfully. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Qwen3.8-27B](/models/qwen/qwen3-8-27b) | 64.8% | Qwen3.8-27B model card | 2026-08 | *Score above is self-reported by Alibaba's Qwen Team on the official Hugging Face model card and shown for context. It is not a Benchgen measurement, and is not drawn from a shared third-party leaderboard — only the 1 score above is currently tracked locally.* ## WebArena-Verified on Benchgen > No Benchgen results yet — [be the first to run WebArena-Verified](/benchmarks/carnegie-mellon-university-servicenow/webarena-verified). ## WebArena-Verified vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **WebArena-Verified** | Realistic, long-horizon browser-use agent control across e-commerce/forum/GitLab/CMS sites | Low | | [AndroidWorld](/benchmarks/google-deepmind/androidworld) | Dynamic, parameterized mobile-use agent control | Low | | [OSWorld-Verified](/benchmarks/osworld/osworld-verified) | Real desktop/web computer-use agent control | — | WebArena-Verified is the browser-use counterpart to device-level agent benchmarks like AndroidWorld (mobile) and OSWorld-Verified (desktop) — use it to assess whether an agent can complete realistic, multi-step tasks purely within a web browser across diverse site categories, independent of mobile or desktop OS control. ## Run WebArena-Verified on Your Model Benchgen lets you evaluate your own model against WebArena-Verified's 812 audited browser-use tasks and track results across model versions. ## Related Models & Benchmarks - Top performers: [Qwen3.8-27B](/models/qwen/qwen3-8-27b) - Related benchmarks: [AndroidWorld](/benchmarks/google-deepmind/androidworld) · [OSWorld-Verified](/benchmarks/osworld/osworld-verified) ## Frequently Asked Questions

What is WebArena-Verified?

WebArena-Verified is an audited, deterministically-scored re-release (ServiceNow, 2025) of the original WebArena benchmark (Zhou et al., CMU, 2023) — a fully functional web environment with 812 realistic tasks across e-commerce, social-forum, GitLab, and content-management sites, used to evaluate autonomous browser-use agents.

What does a good WebArena-Verified score look like?

Qwen3.8-27B reports 64.8% as of August 2026, the only score currently tracked on Benchgen. For context, the original 2023 paper's best GPT-4-based agent achieved only 14.4% end-to-end task success, versus 78.2% human performance — so scores in the 60%+ range reflect substantial progress since release.

Who created WebArena-Verified?

The original WebArena benchmark was created by Shuyan Zhou and co-authors at Carnegie Mellon University (2023); the audited WebArena-Verified re-release was created by Amine El hattami, Megh Thakkar, Nicolas Chapados, and Christopher Pal, presented at the Scaling Environments for Agents (SEA) Workshop at NeurIPS 2025.

Is WebArena-Verified saturated?

No — WebArena-Verified remains a low-saturation benchmark; its 812 realistic, long-horizon browser tasks with deterministic scoring keep meaningful headroom between models.

How does WebArena-Verified differ from AndroidWorld?

WebArena-Verified tests agent control purely within a web browser across e-commerce/forum/GitLab/CMS sites, while AndroidWorld tests multi-step agent control on a real Android device across 20 mobile apps — both verify success against ground-truth state rather than static answer-matching, but for different device/platform scopes.

--- *Benchmark definition paraphrased from [Zhou et al. 2023](https://arxiv.org/abs/2307.13854) (original WebArena) and the [ServiceNow/webarena-verified](https://github.com/ServiceNow/webarena-verified) re-release documentation. State-of-the-art scores sourced from published model cards and attributed inline. Last updated 2026-10-31.*

WebArena-Verified — Results

WebArena-Verified — Evaluations