MRCRv2

MRCRv2

Quick answer: MRCRv2 (Multi-turn Retrieval Correctness and Relevance, version 2) evaluates AI models on accurate information retrieval across long, multi-turn conversation contexts. It tests whether a model can correctly recall and reason over information introduced earlier in extended dialogues. Fugu Ultra scores 93.6% — the highest among evaluated systems — and Fugu scores 86.6% as of June 2026.

At a Glance

What it tests: Long-context retrieval accuracy — whether a model can recall specific facts, instructions, or details from earlier in a multi-turn conversation, even when the context has grown very long.

Why it matters: In agentic and assistant workflows, models must maintain accurate recall across long interaction histories. MRCRv2 directly measures this retrieval fidelity under realistic multi-turn conditions.

Known limitations: Retrieval-focused benchmarks may not capture reasoning quality — a model can retrieve accurately but still fail to reason correctly with retrieved information. Results are sensitive to context window size and attention patterns.

What MRCRv2 Measures

MRCRv2 presents models with extended multi-turn conversation histories containing specific facts, instructions, or details spread across many turns. At test time, the model is queried about information from earlier in the conversation — sometimes from the very beginning, sometimes from the middle — requiring accurate long-range retrieval even when more recent content may create interference.

The benchmark tests both retrieval correctness (is the recalled fact accurate?) and the model's ability to maintain relevant attention across a context window that may span hundreds of thousands of tokens. Scores above 90% indicate reliable long-context recall.

Notably, Fugu Ultra (93.6%) outperforms Fugu (86.6%) by 7 percentage points on this benchmark, suggesting that deeper multi-agent coordination benefits long-context retrieval tasks — likely because Fugu Ultra can use specialist agents for document comprehension.

Benchmark Specifications

State-of-the-Art Results

Scores sourced from Sakana AI's Fugu technical report, June 2026. Note: one baseline model scores 94.8% — likely Fable 5 or Mythos Preview individually; Fugu Ultra at 93.6% is the highest publicly accessible API result.