GAIA2

GAIA2

Quick answer: GAIA2 is Meta FAIR's successor to the original GAIA benchmark, testing AI agents on 800 dynamic scenarios built in the Meta Agents Research Environments (ARE) platform. Unlike static benchmarks, GAIA2 scenarios run asynchronously with time-based events, requiring agents to handle ambiguity, adapt to a changing environment, collaborate with other agents, and operate under temporal constraints.

At a Glance

What it tests: Seven core agent capabilities — Execution, Search, Adaptability, Time, Ambiguity, Agent2Agent collaboration, and robustness to environmental Noise — across 10 simulated "universes" with fictional users, contacts, and apps.

Why it matters: Most agent benchmarks use static sandboxes and final-answer checks; GAIA2 runs asynchronously, surfacing failure modes (missed deadlines, mishandled ambiguity, poor multi-agent coordination) that are invisible in single-turn evaluations.

Known limitations: Scenarios and universes are entirely synthetic (built on Llama 3.3/Llama 4 Maverick outputs), and as a Meta-created and Meta-maintained benchmark, cross-lab reported scores should be read alongside independently-verified results where available.

What GAIA2 Measures

GAIA2 is built on Meta Agents Research Environments (ARE), a platform for constructing simulated, time-aware environments where events unfold dynamically rather than waiting for the agent to act. Each of the 800 scenarios drops an agent into one of 10 pre-populated "universes" — with synthetic contacts, messages, calendars, and apps — and asks it to complete a task such as updating contacts, scheduling conflicting events, or coordinating with another agent.

The benchmark evaluates seven capability categories: Execution (multi-step planning and state changes), Search (information gathering and synthesis), Adaptability (responding to events that change mid-task), Time (temporal reasoning and scheduling), Ambiguity (recognizing and clarifying unclear or impossible requests), Agent2Agent (collaborating with other simulated agents), and Noise (robustness to unreliable apps/environments). Because scenarios can involve events firing on a clock rather than only in response to agent actions, GAIA2 captures failure modes — like an agent that plans correctly but fails to act before a deadline — that static, turn-based benchmarks cannot.

Benchmark Specifications

How GAIA2 Is Scored

GAIA2 reports an overall success rate across all scenarios, plus a per-capability breakdown across the seven core dimensions. Runs are typically repeated 3x per scenario for variance analysis, since asynchronous, time-based tasks can produce different outcomes run to run. The paper's headline finding is that no single system dominates across the full intelligence spectrum — stronger reasoning models often trade off against efficiency, and budget-scaling curves plateau — making GAIA2 a benchmark for well-rounded agent robustness rather than raw model capability alone.

GAIA2 on Benchgen

No Benchgen results yet — be the first to run GAIA2.

GAIA2 vs Other Benchmarks

GAIA2 is the right choice for stress-testing an agent's ability to handle ambiguity, timing, and multi-agent coordination in a simulated but dynamic setting, complementing tool-use benchmarks like MCP-Atlas and real-environment benchmarks like OSWorld-Verified.

Run GAIA2 on Your Model

Benchgen lets teams run GAIA2 against their own model and harness combinations, tracking per-capability scores over time rather than relying on a single vendor-reported snapshot — useful for catching regressions in ambiguity handling or multi-agent coordination introduced by a prompt or scaffold change.

Frequently Asked Questions

Benchmark definition based on Andrews et al., "ARE: Scaling Up Agent Environments and Evaluations" (arXiv:2509.17158, Meta FAIR). Last updated 2026-08-10.