ClawBench
1 phaseActive
Live-website agent benchmark testing everyday browsing/task-completion tasks on real, non-sandboxed sites.
Overview
ClawBench
Quick answer: ClawBench evaluates AI browsing/computer-use agents on everyday tasks performed against real, live, production websites rather than sandboxed clones or offline snapshots — deliberately testing the gap between sandbox performance and the messiness of real-world sites (dynamic layouts, ads, auth flows, rate limits, and unannounced UI changes).
At a Glance
What it tests: Whether an agent can complete common online tasks (search, form-filling, checkout-style flows, information lookup) against unmodified, live production websites.
Why it matters: Many web-agent benchmarks run against static clones or sandboxed replicas of websites, which understate real-world difficulty. ClawBench's live-site design surfaces failure modes — layout drift, anti-bot measures, session/auth handling — that only appear outside a controlled sandbox.
Known limitations: Because it targets live sites, task difficulty and even task validity can shift over time as target websites change, making long-term score comparability harder than with a static benchmark. Detailed public leaderboard scores were not available at the time of writing.
What ClawBench Measures
ClawBench departs from the sandboxed-website convention used by most browsing-agent benchmarks (which typically run against frozen clones or offline replicas for reproducibility). Instead, it sends agents against real, currently-live production websites to complete everyday tasks — the kind of search, lookup, and form-based interactions a human would routinely perform online. This design choice trades some reproducibility for realism: agents must handle the actual complexity of production web pages, including dynamic content loading, authentication flows, anti-automation defenses, and UI changes that a static sandbox would never expose.
The benchmark's premise is that a meaningful gap exists between how agents perform on curated, sandboxed website benchmarks and how they perform "in the wild" — and that gap is exactly what matters for teams deploying browsing agents in production, where the environment is never frozen in time.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Agent (live web browsing / computer use) |
| Metric | Task success rate (%) |
| Environment | Real, live production websites (not sandboxed clones) |
| Saturation | Low |
| Created by | Agent benchmarking research community, 2026 |
| Paper | Not yet publicly attributed to a single paper |
| Dataset | Not public |
How ClawBench Is Scored
Each task is scored as successfully completed or not, based on whether the agent achieves the specified real-world outcome (e.g., locating specific information, completing a form, or navigating to a required page) on the live target site. Because the underlying websites can change between evaluation runs, ClawBench scores are best interpreted as a snapshot of agent robustness at a point in time rather than a permanently fixed leaderboard.
ClawBench on Benchgen
No Benchgen results yet — be the first to run ClawBench.
ClawBench vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| ClawBench | Everyday tasks on live, unmodified production websites | Low |
| OSWorld-Verified | Computer-use / GUI agent tasks (sandboxed) | Low |
| ClawMark | Long-horizon, multi-day "coworker" agent tasks | Low |
| MCP Atlas | MCP tool-use in agentic workflows | Low |
ClawBench and OSWorld-Verified both test computer-use/browsing agents, but OSWorld-Verified runs in a controlled, reproducible sandbox while ClawBench deliberately targets live, uncontrolled production sites. Use ClawBench when you need signal on how an agent handles the unpredictability of real websites, not just a controlled replica.
Run ClawBench on Your Model
Benchgen lets teams evaluate their own browsing/computer-use agents against realistic, evolving web environments and track robustness over time as target sites change — rather than relying solely on static sandbox scores that may overstate real-world readiness.
Related Models & Benchmarks
- Related benchmarks: OSWorld-Verified · ClawMark · MCP Atlas