SciCode

SciCode

Quick answer: SciCode (Tian et al., 2024) is a scientific programming benchmark that presents AI models with real research problems from physics, mathematics, chemistry, and biology, requiring multi-step code implementations. Unlike competitive programming benchmarks, SciCode tasks involve domain-specific scientific knowledge. Fugu scores 60.1% and Fugu Ultra scores 58.7% as of June 2026.

At a Glance

What it tests: Multi-step scientific programming — implementing research algorithms, numerical methods, and scientific simulations across physics, math, chemistry, and biology domains.

Why it matters: SciCode bridges coding and scientific reasoning, testing whether a model can translate domain-specific knowledge (e.g., solving differential equations, implementing a simulation) into working code — closer to how AI systems are actually used in research.

Known limitations: Requires both programming skill and domain-specific scientific knowledge; a model that excels at competitive programming may score lower here due to domain knowledge gaps. Task coverage across scientific domains is uneven.

What SciCode Measures

SciCode sources problems directly from research papers and textbooks, requiring models to implement scientific algorithms and methods from scratch. Each task involves writing code to solve a specific scientific computation — which must pass correctness tests that verify the numerical or logical output. Tasks span multiple scientific disciplines and typically require several implementation steps with domain-specific understanding.

This makes SciCode harder than purely algorithmic coding benchmarks: a model that can write clean algorithmic code but lacks physics or chemistry knowledge will fail on many tasks. Scores in the 55–65% range represent frontier performance as of mid-2026.

Benchmark Specifications

State-of-the-Art Results

Scores sourced from Sakana AI's Fugu technical report, June 2026. Note that Fugu (standard) slightly outperforms Fugu Ultra on this benchmark, suggesting that latency-optimised orchestration may be more effective for structured scientific coding tasks.