# ProgramBench ![Category](https://img.shields.io/badge/Category-Coding-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Accuracy-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Low-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) [![Website](https://img.shields.io/badge/Website-vals.ai%2Fbenchmarks%2Fprogrambench-0969da)](https://www.vals.ai/benchmarks/programbench) > **Quick answer:** ProgramBench is a real-world programming benchmark maintained by Vals AI, an independent model-evaluation platform. It tests models on practical coding tasks with an emphasis on correctness and real-world applicability rather than isolated algorithmic puzzles. Kimi K3 scores 77.8% as of July 2026. ## At a Glance **What it tests:** A model's ability to produce correct, working code for practical programming tasks, evaluated by an independent third-party leaderboard. **Why it matters:** Vals AI runs standardized, independently-administered evaluations across many frontier models, making ProgramBench a useful cross-vendor reference point that isn't self-reported by any single lab. **Known limitations:** As a third-party leaderboard benchmark, detailed task composition and scoring methodology are less publicly documented than academic benchmarks with published papers. ## What ProgramBench Measures ProgramBench evaluates models on real-world programming tasks curated and administered by Vals AI, an independent AI evaluation platform. The benchmark emphasizes practical, applied coding correctness rather than narrow algorithmic puzzle-solving, and results are published on Vals AI's public leaderboard alongside many other frontier models for direct comparison. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Coding | | Metric | % accuracy | | Saturation | Low | | Created by | Vals AI | | Leaderboard | [vals.ai/benchmarks/programbench](https://www.vals.ai/benchmarks/programbench) | ## How ProgramBench Is Scored Models generate solutions for each programming task, which are evaluated for correctness (e.g., via test execution or reference-solution comparison), producing an aggregate % accuracy score published on the Vals AI leaderboard. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Kimi K3](/models/moonshot/kimi-k3) | 77.8% | [Kimi K3 technical report](https://www.kimi.com/blog/kimi-k3) | 2026-07 | *Score sourced from Moonshot AI's Kimi K3 announcement, July 2026.* ## ProgramBench on Benchgen > No Benchgen results yet — [be the first to run ProgramBench](/benchmarks/programbench). ## ProgramBench vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **ProgramBench** | Real-world programming tasks (Vals AI) | Low | | [BigCodeBench](/benchmarks/bigcodebench) | Practical code generation | Medium | | [LiveCodeBench](/benchmarks/livecodebench) | Contamination-resistant competitive coding | Medium | | [DeepSWE](/benchmarks/deepswe) | Software engineering agent tasks | Low | ## Run ProgramBench on Your Model Benchgen lets you run ProgramBench-style evaluations against your own model, tracking coding accuracy over time as you fine-tune or update your deployment. ## Frequently Asked Questions

What is ProgramBench?

ProgramBench is a real-world programming benchmark maintained by Vals AI, an independent AI model evaluation platform, testing practical coding correctness across frontier models.

What does a good score look like on ProgramBench?

Kimi K3 reports 77.8% as of July 2026, a strong result among current frontier coding models on this independently-administered benchmark.

Who created ProgramBench?

ProgramBench is created and maintained by Vals AI, an independent AI evaluation company that publishes standardized benchmark results across many frontier models.

Rank	Model	Score
1	kimi-k3	77.8
2	deepseek-v4-1-flash	20.3
3	hy4-preview	17.5

ProgramBench — Results

ProgramBench — Evaluations