SUPERChem — Results
| Rank | Model | Score |
|---|---|---|
| 1 | hy4-preview | 66.4 |
SUPERChem
1 phaseActive
PKU's deep chemistry-reasoning benchmark — 500 expert-curated problems with process-level Reasoning Path Fidelity scoring, text and multimodal forms.
Overview
SUPERChem
Quick answer: SUPERChem is a deep chemistry-reasoning benchmark from Peking University researchers — 500 expert-curated problems spanning multiple chemistry subfields, in matched text-only and multimodal forms, scored on both final-answer accuracy and process-level Reasoning Path Fidelity (RPF).
At a Glance
What it tests: Deep, multi-step chemical reasoning across subfields, evaluated not just on the final answer but on whether the model's reasoning process matches an expert solution path.
Why it matters: Getting the right chemistry answer by a flawed reasoning process doesn't generalize — RPF specifically rewards models whose intermediate reasoning is sound, not just lucky.
Known limitations: RPF scoring requires comparing against an expert-authored reasoning DAG, which is more complex to reproduce independently than a simple final-answer check.
What SUPERChem Measures
SUPERChem was built by a Peking University-affiliated research team (led by Zehua Zhao) to test deep chemical reasoning across subfields, using 500 expert-curated problems available in both text-only and multimodal forms. Each problem ships with an expert-authored solution path, enabling process-level assessment via a metric the authors call Reasoning Path Fidelity (RPF) — a comparison between the model's own reasoning representation and the expert's solution DAG (directed acyclic graph of reasoning steps).
A high score on both final-answer accuracy and RPF indicates a model isn't just pattern-matching to the right numeric or categorical answer, but is actually reasoning through the underlying chemistry correctly — a stronger signal of genuine domain competence than accuracy alone.
Benchmark Specifications
| Field | Value |
|---|---|
| Task category | Reasoning (chemistry) |
| Metric | Final-answer accuracy + Reasoning Path Fidelity (RPF) |
| Number of tasks | 500 |
| Saturation | Low |
| Created by | Zehua Zhao et al. (Peking University) |
| Source paper | Zhao et al. 2025 |
| GitHub | catalystforyou/SUPERChem_eval |
| Dataset | Hugging Face |
How SUPERChem Is Scored
Models are graded on final-answer correctness (accuracy) and, separately, on Reasoning Path Fidelity — a structural comparison between the model's reasoning trace and an expert-authored solution DAG for the same problem. Reported scores in vendor benchmark tables are typically the accuracy metric.
State-of-the-Art Results
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Hy4 Preview | 66.4% | Tencent Hunyuan model card | 2026-08 |
Scores sourced from published technical reports and model cards. Results depend on harness, prompt format, and effort settings — see each source for methodology.
SUPERChem on Benchgen
No Benchgen results yet — be the first to run SUPERChem.
SUPERChem vs Other Benchmarks
| Benchmark | What it tests | Tasks | Saturation |
|---|---|---|---|
| SUPERChem | Deep chemical reasoning + process fidelity | 500 | Low |
| GPQA Diamond | Graduate-level science (bio/chem/physics) | 198 | Medium |
| CritPt | Research-level physics reasoning | — | Low |
Use SUPERChem when you need chemistry-specific depth and process-level reasoning assessment, beyond the broader multi-domain science coverage of GPQA Diamond.
Run SUPERChem on Your Model
Benchgen lets teams run SUPERChem against their own model versions, compare results across runs, and catch regressions in chemical reasoning quality — rather than relying on a single vendor-reported number.
Related Models & Benchmarks
- Top performers: Hy4 Preview
- Related benchmarks: GPQA Diamond · CritPt