LiveCodeBench

LiveCodeBench

Quick answer: LiveCodeBench is a contamination-resistant competitive programming benchmark by Jain et al. (2024) that continuously collects fresh problems from LeetCode, AtCoder, and Codeforces after a specified cutoff date. Because the problems post-date training data, scores reflect genuine coding ability rather than memorised solutions. Fugu Ultra scores 93.2% and Fugu scores 92.9% as of June 2026.

At a Glance

What it tests: Competitive programming ability — generating correct code solutions to algorithmic problems drawn from live contests after model training cutoffs.

Why it matters: Most coding benchmarks suffer from contamination (problems appear in training data); LiveCodeBench's rolling fresh-problem design provides a more honest measure of a model's actual coding skill.

Known limitations: Competitive programming skill correlates with but is not identical to practical software engineering ability. Problem difficulty is bounded by contest problem styles.

What LiveCodeBench Measures

LiveCodeBench collects problems from three competitive programming platforms — LeetCode, AtCoder, and Codeforces — posted after a specified cutoff date that post-dates model training. Models must generate a correct program solution; a solution is counted as passing if it produces the correct output on all test cases.

By harvesting problems from after training cutoffs, LiveCodeBench directly addresses the contamination problem that plagues static benchmarks like HumanEval, where top models have effectively memorised the test set. The benchmark is updated continuously, making it a living measure of coding progress. As of 2026, scores above 90% represent frontier-class coding performance.

Benchmark Specifications

How LiveCodeBench Is Scored

A solution passes if it produces the correct output on all provided test cases (pass@1 — no retries). Score is the percentage of problems where the model's first attempt passes all tests. The benchmark uses a rolling window of problems, and results may vary depending on which time window is evaluated — always check the source report for the specific date range used.

State-of-the-Art Results

Scores sourced from Sakana AI's Fugu technical report, June 2026. LiveCodeBench scores depend on the time window of problems evaluated — see source for exact window.