ClawMark
1 phaseActive
Long-horizon, multi-day agent benchmark simulating persistent 'coworker' tasks with drifting external state.
Overview
ClawMark
Quick answer: ClawMark evaluates AI agents in long-horizon "coworker" settings — multi-turn, multi-day, multimodal tasks involving persistent, tool-backed services and external state that keeps drifting between sessions, rather than the single-session, static-environment tasks most agent benchmarks use. It pushes past static-task evaluation toward realistic, stateful, long-horizon scenarios that mirror how an agent would actually be embedded in an ongoing workflow.
At a Glance
What it tests: An agent's ability to operate as a persistent "coworker" across multiple sessions spanning days, maintaining task continuity and correctly handling external state that changes independently of the agent's own actions.
Why it matters: Most agent benchmarks evaluate a single session against a static environment. Real deployed agents — a coding assistant, a research assistant, an ops agent — operate across many sessions over days or weeks, in environments that change for reasons unrelated to the agent (new emails arrive, files get edited by others, APIs return different data). ClawMark is designed specifically to test that continuity and drift-handling.
Known limitations: Long-horizon, multi-day benchmarks are inherently harder to run and reproduce than single-session ones, and detailed public leaderboard scores were not available at the time of writing.
What ClawMark Measures
ClawMark constructs scenarios that unfold over multiple sessions — potentially spanning simulated days — where the agent must pick up context from earlier sessions, use persistent, tool-backed services (not stateless mock tools), and adapt to state that changes between sessions independently of anything the agent did. This "drifting external state" design is the benchmark's key differentiator: rather than a frozen environment waiting for the agent to act, ClawMark's environment keeps moving, exactly as a real inbox, codebase, or shared document would.
The benchmark also incorporates multimodal and multi-turn elements, reflecting that realistic long-horizon "coworker" tasks rarely stay confined to a single modality or a single, uninterrupted conversation. Success requires the agent to maintain accurate memory of prior sessions, distinguish stale context from updated state, and re-plan around changes it didn't cause.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Agent (long-horizon, multi-day, multimodal) |
| Metric | Task success rate (%) |
| Environment | Persistent, tool-backed services with drifting external state |
| Session structure | Multi-turn, multi-day |
| Saturation | Low |
| Created by | Agent benchmarking research community, 2026 |
| Paper | Not yet publicly attributed to a single paper |
| Dataset | Not public |
How ClawMark Is Scored
Each multi-day scenario is scored on whether the agent successfully completes the overall goal across all its constituent sessions, factoring in whether the agent correctly recognized and adapted to state changes that occurred between sessions. An agent that succeeds within a single session but fails to notice or adapt to drifted state in a later session is scored as failing the scenario, since state-drift handling is central to what ClawMark is designed to measure.
ClawMark on Benchgen
No Benchgen results yet — be the first to run ClawMark.
ClawMark vs Other Benchmarks
| Benchmark | What it tests | Saturation |
|---|---|---|
| ClawMark | Multi-day, multi-turn "coworker" tasks with drifting state | Low |
| ClawBench | Everyday tasks on live, unmodified production websites | Low |
| SWE-Marathon | Long-horizon software engineering task completion | Low |
| StaminaBench | Consecutive change-request turns survived (single session) | Low |
ClawMark, StaminaBench, and SWE-Marathon all push past single-shot task evaluation, but ClawMark is distinguished by explicitly modeling state drift between sessions (days, not just turns), rather than testing endurance within one continuous session.
Run ClawMark on Your Model
Benchgen lets teams evaluate their own agents across multi-session, long-horizon workflows with realistic state drift — surfacing continuity and re-planning failures that single-session benchmarks would never catch before they reach production deployments.
Related Models & Benchmarks
- Related benchmarks: ClawBench · SWE-Marathon · StaminaBench