# AetherCode ![Category](https://img.shields.io/badge/Category-Competitive%20Programming-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Score%20(0--1)-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2025-555) > **Quick answer:** AetherCode is ByteDance's benchmark of olympiad-level competitive programming problems from IOI/ICPC contests, testing LLMs on advanced algorithmic reasoning and code synthesis. Best Pass@1: 35.5% (o4-mini-high, 2025). Even top models solve fewer than 40% of problems. ## At a Glance **What it tests:** An LLM's ability to solve hard algorithmic coding problems drawn from competitive programming contests — requiring mastery of data structures, graph algorithms, dynamic programming, number theory, and combinatorics. **Why it matters:** Standard coding benchmarks (HumanEval, MBPP, LiveCodeBench) have saturated for frontier models. AetherCode's olympiad-difficulty floor creates meaningful separation between models that can reason through complex multi-step algorithmic problems and those that cannot, making it relevant for evaluating state-of-the-art reasoning-heavy code generation. **Known limitations:** No paper or public dataset is available as of mid-2026, limiting independent reproduction and analysis. With only 2 models tracked publicly, leaderboard coverage is sparse. ## What AetherCode Measures AetherCode focuses on the highest tier of coding difficulty: problems from programming olympiads, where solutions require not just syntactically correct code but deep algorithmic insight. Unlike benchmarks built from interview-style questions or library usage tasks, olympiad problems demand that a model understand and apply non-trivial algorithms correctly under problem-specific constraints. The benchmark evaluates whether a model can derive the correct algorithmic approach, implement it efficiently, and handle edge cases — all in a single pass without interactive debugging. This makes it a direct measure of end-to-end algorithmic problem-solving rather than code autocomplete or library API recall. Scores are reported on a 0–1 scale, reflecting the fraction of problems solved correctly. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Competitive programming / Coding | | Metric | Score (0–1) | | Difficulty | Olympiad level | | Languages | English | | Saturation | Low | | Created by | ByteDance | | Paper | Not yet public | | GitHub | Not yet public | | Dataset | Not yet public | ## How AetherCode Is Scored Models are evaluated by executing their generated code against hidden test cases for each problem. A problem is marked correct only if the solution passes all test cases. The aggregate score is the fraction of problems solved, reported on a 0–1 scale where 1.0 means all olympiad problems solved correctly. ## AetherCode on Benchgen > No Benchgen results yet — [be the first to run AetherCode](/benchmarks/bytedance/aethercode). ## AetherCode vs Other Benchmarks | Benchmark | What it tests | Difficulty | Saturation | |---|---|---|---| | **AetherCode** | Olympiad competitive programming | Highest | Low | | [LiveCodeBench Pro](/benchmarks/livecodebench-team/livecodebench-pro) | Competitive programming (continuous) | Very high | Low | | [SWE-Bench Pro](/benchmarks/swe-bench-team/swe-bench-pro) | Real-world software engineering | High | Low | | [SciCode](/benchmarks/tian-et-al/scicode) | Scientific research coding | High | Low | AetherCode sits at the hardest end of the coding difficulty spectrum alongside LiveCodeBench Pro. Use AetherCode when you need to differentiate frontier models on pure algorithmic reasoning; use SWE-Bench Pro when production engineering tasks matter more. ## Run AetherCode on Your Model Benchgen lets you run AetherCode against your own model checkpoints and track performance across fine-tuning runs, catching regressions in algorithmic reasoning before they reach production. Version-controlled benchmark results replace one-off comparisons with a continuous quality signal. ## Frequently Asked Questions

What is AetherCode?

AetherCode is a competitive programming benchmark from ByteDance that evaluates LLMs on olympiad-level algorithmic coding problems. It targets the hardest tier of code generation, requiring mastery of advanced data structures and algorithms.

What does a good score look like on AetherCode?

Scores are reported on a 0–1 scale. As of mid-2026, the highest publicly tracked score is 0.679 (Seed 2.1 Turbo), indicating the benchmark remains well below saturation and that even frontier models solve fewer than 70% of olympiad problems.

Who created AetherCode?

AetherCode was created by ByteDance. No public paper or dataset has been released as of mid-2026.

Rank	Model	Score
1	seed-2-1-turbo	0.679
2	seed-2-1-pro	0.658
3	o4-mini-high	0.355
4	gemini-2-5-pro	0.327
5	seed-1-6-thinking	0.266
6	deepseek-r1-0528	0.223
7	gemini-2-5-flash	0.203
8	qwen3-235b-a22b	0.176
9	qwen3-32b	0.163
10	gpt-4-1	0.105
11	qwen3-8b	0.1
12	kimi-k2-instruct	0.098
13	deepseek-v3-0324	0.085
14	qwen3-coder-480b-a35b	0.077
15	gpt-4o	0.044

AetherCode — Results

AetherCode — Evaluations