# SEC-Bench Pro ![Category](https://img.shields.io/badge/Category-Agent%2FCybersecurity-1f6feb) ![Metric](https://img.shields.io/badge/Metric-%25%20Task%20Success-2da44e) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) ![Created](https://img.shields.io/badge/Created-2026-555) > **Quick answer:** SEC-Bench Pro evaluates AI agents on professional-grade defensive security engineering tasks — the kind of secure code review and vulnerability-remediation work security teams do day to day, as distinct from offensive exploit-development benchmarks like ExploitBench. It first appeared publicly in OpenAI's GPT-6 Astra announcement (September 2026), where Astra scored 85.4%, ahead of GPT-5.6 Sol's 79.1%. ## At a Glance **What it tests:** Professional defensive-security workflows — secure code review, vulnerability identification and remediation, and related engineering tasks a security team would handle in production, rather than offensive exploit construction. **Why it matters:** Most public cybersecurity benchmarks (ExploitBench, ExploitGym) focus on offensive capability — can a model build a working exploit. SEC-Bench Pro instead measures the defensive counterpart: can a model be trusted to do the security engineering work that keeps systems safe. This distinction matters for responsible deployment, since offensive and defensive capability don't necessarily scale together. **Known limitations:** Full methodology (task count, dataset composition, public availability) has not yet been documented in an independent paper as of this writing — currently known primarily through frontier lab technical reports. ## What SEC-Bench Pro Measures SEC-Bench Pro evaluates models on professional-tier security engineering tasks — the practical, defensive work of a security team, such as secure code review and patching, as opposed to constructing novel exploits against hardened targets. This makes it a natural complement to offensive-capability benchmarks like ExploitBench and ExploitGym: a model could plausibly score very differently on offensive versus defensive security tasks, and tracking both gives a fuller picture of a model's net effect on the security landscape. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Agent / Cybersecurity (defensive) | | Metric | % task success | | Saturation | Medium | | Created by | OpenAI (introduced via GPT-6 Astra announcement) | | Public dataset | Not yet publicly released as of Sep 2026 | ## How SEC-Bench Pro Is Scored Scores are reported as a percentage task-success rate across the benchmark's professional security-engineering task set. Exact grading criteria and task composition have not been fully disclosed publicly as of this writing. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [GPT-6 Astra](/models/openai/gpt-6-astra) | 85.4% | [OpenAI: GPT-6 Astra](https://openai.com/index/gpt-6-astra/) | 2026-09 | | 2 | [GPT-5.6 Sol](/models/openai/gpt-5-6-sol) | 79.1% | [OpenAI: GPT-6 Astra](https://openai.com/index/gpt-6-astra/) | 2026-09 | *Scores sourced from OpenAI's GPT-6 Astra announcement (September 2026).* ## SEC-Bench Pro on Benchgen > No Benchgen results yet — [be the first to run SEC-Bench Pro](/benchmarks/sec-bench-pro). ## SEC-Bench Pro vs Other Benchmarks | Benchmark | What it tests | Saturation | |---|---|---| | **SEC-Bench Pro** | Defensive security engineering (code review, remediation) | Medium | | [ExploitBench](/benchmarks/seunghyun-lee-david-brumley/exploitbench) | Offensive exploit construction (V8 bugs) | Low | | [ExploitGym](/benchmarks/zhun-wang-nico-schiller-et-al-uc-berkeley/exploitgym) | Offensive exploit construction (broader gym) | Low | | [SRE-Bench](/benchmarks/sre-bench) | Reverse engineering / binary analysis | Low | SEC-Bench Pro fills the defensive-security gap alongside the mostly-offensive existing cybersecurity benchmark set — important for evaluating net security impact, not just raw exploit capability. ## Run SEC-Bench Pro on Your Model Benchgen lets teams track defensive security-engineering performance across model versions, catching regressions a single vendor-reported snapshot would miss. ## Related Models & Benchmarks - Top performers: [GPT-6 Astra](/models/openai/gpt-6-astra) · [GPT-5.6 Sol](/models/openai/gpt-5-6-sol) - Related benchmarks: [ExploitBench](/benchmarks/seunghyun-lee-david-brumley/exploitbench) · [ExploitGym](/benchmarks/zhun-wang-nico-schiller-et-al-uc-berkeley/exploitgym) · [SRE-Bench](/benchmarks/sre-bench) ## Frequently Asked Questions

What is SEC-Bench Pro?

A benchmark testing AI agents on professional defensive security engineering tasks — secure code review and vulnerability remediation — introduced publicly via OpenAI's GPT-6 Astra announcement in September 2026.

What does a good SEC-Bench Pro score look like?

As of September 2026, frontier models score in the high 70s to mid 80s percent, with GPT-6 Astra leading at 85.4%.

Who created SEC-Bench Pro?

SEC-Bench Pro was introduced by OpenAI, first publicly referenced in the GPT-6 Astra model announcement (September 2026).

How does SEC-Bench Pro differ from ExploitBench?

ExploitBench measures offensive exploit construction against hardened real-world bugs. SEC-Bench Pro measures the defensive counterpart — secure code review and remediation work.

--- *Benchmark definition based on OpenAI's GPT-6 Astra announcement. State-of-the-art scores sourced from the same and attributed inline. Last updated 2026-09-07.*

Rank	Model	Score
1	gpt-6-astra	85.4
2	deepseek-v4-1-flash	62.8
3	deepseek-v4-1-flash	62.8

SEC-Bench Pro — Results

SEC-Bench Pro — Evaluations