# GAIA2 ![Category](https://img.shields.io/badge/Category-Agent-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Success-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-800-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-Sep%202025-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2509.17158) [![GitHub](https://img.shields.io/badge/GitHub-facebookresearch%2Fmeta--agents--research--environments-181717?logo=github&logoColor=white)](https://github.com/facebookresearch/meta-agents-research-environments) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/meta-agents-research-environments/gaia2) > **Quick answer:** GAIA2 is Meta FAIR's successor to the original GAIA benchmark, testing AI agents on 800 dynamic scenarios built in the Meta Agents Research Environments (ARE) platform. Unlike static benchmarks, GAIA2 scenarios run asynchronously with time-based events, requiring agents to handle ambiguity, adapt to a changing environment, collaborate with other agents, and operate under temporal constraints. ## At a Glance **What it tests:** Seven core agent capabilities — Execution, Search, Adaptability, Time, Ambiguity, Agent2Agent collaboration, and robustness to environmental Noise — across 10 simulated "universes" with fictional users, contacts, and apps. **Why it matters:** Most agent benchmarks use static sandboxes and final-answer checks; GAIA2 runs asynchronously, surfacing failure modes (missed deadlines, mishandled ambiguity, poor multi-agent coordination) that are invisible in single-turn evaluations. **Known limitations:** Scenarios and universes are entirely synthetic (built on Llama 3.3/Llama 4 Maverick outputs), and as a Meta-created and Meta-maintained benchmark, cross-lab reported scores should be read alongside independently-verified results where available. ## What GAIA2 Measures GAIA2 is built on Meta Agents Research Environments (ARE), a platform for constructing simulated, time-aware environments where events unfold dynamically rather than waiting for the agent to act. Each of the 800 scenarios drops an agent into one of 10 pre-populated "universes" — with synthetic contacts, messages, calendars, and apps — and asks it to complete a task such as updating contacts, scheduling conflicting events, or coordinating with another agent. The benchmark evaluates seven capability categories: Execution (multi-step planning and state changes), Search (information gathering and synthesis), Adaptability (responding to events that change mid-task), Time (temporal reasoning and scheduling), Ambiguity (recognizing and clarifying unclear or impossible requests), Agent2Agent (collaborating with other simulated agents), and Noise (robustness to unreliable apps/environments). Because scenarios can involve events firing on a clock rather than only in response to agent actions, GAIA2 captures failure modes — like an agent that plans correctly but fails to act before a deadline — that static, turn-based benchmarks cannot. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent | | Metric | % overall success rate (plus per-capability breakdown) | | Number of tasks | 800 scenarios (160 per core capability, plus augmentation configs) | | Saturation | Low | | Created by | Andrews et al. (Meta FAIR) | | Source paper | [Andrews et al. 2025](https://arxiv.org/abs/2509.17158) | | GitHub | [facebookresearch/meta-agents-research-environments](https://github.com/facebookresearch/meta-agents-research-environments) | | Dataset | [meta-agents-research-environments/gaia2](https://huggingface.co/datasets/meta-agents-research-environments/gaia2) | ## How GAIA2 Is Scored GAIA2 reports an overall success rate across all scenarios, plus a per-capability breakdown across the seven core dimensions. Runs are typically repeated 3x per scenario for variance analysis, since asynchronous, time-based tasks can produce different outcomes run to run. The paper's headline finding is that no single system dominates across the full intelligence spectrum — stronger reasoning models often trade off against efficiency, and budget-scaling curves plateau — making GAIA2 a benchmark for well-rounded agent robustness rather than raw model capability alone. ## GAIA2 on Benchgen > No Benchgen results yet — [be the first to run GAIA2](/benchmarks/gaia2). ## GAIA2 vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **GAIA2** | **Dynamic, time-aware, multi-agent scenarios across 7 capabilities** | **800** | **Low** | | [MCP-Atlas](/benchmarks/mcp-atlas) | Real MCP server tool-use tasks | 1,000 | Low | | [OSWorld-Verified](/benchmarks/osworld-verified) | Real desktop/web computer-use tasks | 369 | Low | GAIA2 is the right choice for stress-testing an agent's ability to handle ambiguity, timing, and multi-agent coordination in a simulated but dynamic setting, complementing tool-use benchmarks like MCP-Atlas and real-environment benchmarks like OSWorld-Verified. ## Run GAIA2 on Your Model Benchgen lets teams run GAIA2 against their own model and harness combinations, tracking per-capability scores over time rather than relying on a single vendor-reported snapshot — useful for catching regressions in ambiguity handling or multi-agent coordination introduced by a prompt or scaffold change. ## Frequently Asked Questions

What is GAIA2?

GAIA2 is Meta FAIR's successor to the original GAIA benchmark, testing AI agents on 800 dynamic, time-aware scenarios across seven capability categories, built on the Meta Agents Research Environments platform.

What does a good GAIA2 score look like?

Scores vary substantially by capability category; frontier agentic models typically land in the 35–50% overall success range, with no system currently dominating across all seven capabilities.

Who created GAIA2?

GAIA2 was created by Meta's AI Research (FAIR) team, led by authors including Romain Froger and Pierre Andrews, and released alongside the Meta Agents Research Environments (ARE) platform in September 2025.

How does GAIA2 differ from the original GAIA?

The original GAIA (2023) used static, single-turn real-world questions. GAIA2 runs asynchronously in simulated environments with dynamic events, adding time constraints, ambiguity handling, and multi-agent collaboration that GAIA didn't test.

Is GAIA2 saturated?

No — GAIA2 remains far from saturated; the paper's own results show budget-scaling curves plateauing well below full success, with meaningful headroom across all seven capability categories.

--- *Benchmark definition based on Andrews et al., "ARE: Scaling Up Agent Environments and Evaluations" (arXiv:2509.17158, Meta FAIR). Last updated 2026-08-10.*

GAIA2 — Results

GAIA2 — Evaluations