BigCodeBench-Full — Results
| Rank | Model | Score |
|---|---|---|
| 1 | qwen2-5-coder-32b-instruct | 0.496 |
BigCodeBench-Full
1 phaseActive
Comprehensive code generation benchmark with 1,140 fine-grained tasks across 7 domains requiring multi-library function calls and complex instruction following.
Overview
BigCodeBench-Full
Quick answer: BigCodeBench-Full (Zhuo et al., 2024) is a comprehensive code generation benchmark evaluating LLMs on 1,140 practical programming tasks across 7 domains using function calls from 139 libraries. It challenges models with complex instructions requiring multi-library tool use.
What is BigCodeBench-Full?
BigCodeBench-Full evaluates practical programming task completion using function calls from 139 Python libraries across 7 domains including web programming, data science, and system tasks. Tasks require multi-function composition and complex instruction following beyond simple algorithmic problems.
Benchmark Details
| Property | Value |
|---|---|
| Tasks | 1,140 programming tasks |
| Libraries | 139 Python libraries |
| Domains | 7 (web, data science, OS, etc.) |
| Metric | Pass rate (test execution) |
| Language | Python |
Source: Zhuo et al. 2024. Last updated 2026-07-24.