# SUPERChem ![Category](https://img.shields.io/badge/Category-Reasoning-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Accuracy%20%2B%20RPF-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-500-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-low-d29922) ![Created](https://img.shields.io/badge/Created-2025-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2512.01274) [![GitHub](https://img.shields.io/badge/GitHub-catalystforyou%2FSUPERChem__eval-181717?logo=github&logoColor=white)](https://github.com/catalystforyou/SUPERChem_eval) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/ZehuaZhao/SUPERChem) > **Quick answer:** SUPERChem is a deep chemistry-reasoning benchmark from Peking University researchers — 500 expert-curated problems spanning multiple chemistry subfields, in matched text-only and multimodal forms, scored on both final-answer accuracy and process-level Reasoning Path Fidelity (RPF). ## At a Glance **What it tests:** Deep, multi-step chemical reasoning across subfields, evaluated not just on the final answer but on whether the model's reasoning process matches an expert solution path. **Why it matters:** Getting the right chemistry answer by a flawed reasoning process doesn't generalize — RPF specifically rewards models whose intermediate reasoning is sound, not just lucky. **Known limitations:** RPF scoring requires comparing against an expert-authored reasoning DAG, which is more complex to reproduce independently than a simple final-answer check. ## What SUPERChem Measures SUPERChem was built by a Peking University-affiliated research team (led by Zehua Zhao) to test deep chemical reasoning across subfields, using 500 expert-curated problems available in both text-only and multimodal forms. Each problem ships with an expert-authored solution path, enabling process-level assessment via a metric the authors call Reasoning Path Fidelity (RPF) — a comparison between the model's own reasoning representation and the expert's solution DAG (directed acyclic graph of reasoning steps). A high score on both final-answer accuracy and RPF indicates a model isn't just pattern-matching to the right numeric or categorical answer, but is actually reasoning through the underlying chemistry correctly — a stronger signal of genuine domain competence than accuracy alone. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Reasoning (chemistry) | | Metric | Final-answer accuracy + Reasoning Path Fidelity (RPF) | | Number of tasks | 500 | | Saturation | Low | | Created by | Zehua Zhao et al. (Peking University) | | Source paper | [Zhao et al. 2025](https://arxiv.org/abs/2512.01274) | | GitHub | [catalystforyou/SUPERChem_eval](https://github.com/catalystforyou/SUPERChem_eval) | | Dataset | [Hugging Face](https://huggingface.co/datasets/ZehuaZhao/SUPERChem) | ## How SUPERChem Is Scored Models are graded on final-answer correctness (accuracy) and, separately, on Reasoning Path Fidelity — a structural comparison between the model's reasoning trace and an expert-authored solution DAG for the same problem. Reported scores in vendor benchmark tables are typically the accuracy metric. ## State-of-the-Art Results | Rank | Model | Score | Source | Date | |---|---|---|---|---| | 1 | [Hy4 Preview](/models/hungyuan/hy4-preview) | 66.4% | [Tencent Hunyuan model card](https://huggingface.co/tencent/Hy4-preview) | 2026-08 | *Scores sourced from published technical reports and model cards. Results depend on harness, prompt format, and effort settings — see each source for methodology.* ## SUPERChem on Benchgen > No Benchgen results yet — [be the first to run SUPERChem](/benchmarks/zehua-zhao-et-al-pku/superchem). ## SUPERChem vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **SUPERChem** | Deep chemical reasoning + process fidelity | 500 | Low | | [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) | Graduate-level science (bio/chem/physics) | 198 | Medium | | [CritPt](/benchmarks/zhu-et-al/critpt) | Research-level physics reasoning | — | Low | Use SUPERChem when you need chemistry-specific depth and process-level reasoning assessment, beyond the broader multi-domain science coverage of GPQA Diamond. ## Run SUPERChem on Your Model Benchgen lets teams run SUPERChem against their own model versions, compare results across runs, and catch regressions in chemical reasoning quality — rather than relying on a single vendor-reported number. ## Related Models & Benchmarks - Top performers: [Hy4 Preview](/models/hungyuan/hy4-preview) - Related benchmarks: [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) · [CritPt](/benchmarks/zhu-et-al/critpt) ## Frequently Asked Questions

What is SUPERChem?

SUPERChem is a deep chemistry-reasoning benchmark of 500 expert-curated problems, scored on both final-answer accuracy and Reasoning Path Fidelity (RPF), created by Zehua Zhao et al. at Peking University.

What does a good SUPERChem score look like?

As of August 2026, frontier models score in the 58–77% range on accuracy; scores above 66% represent strong chemical reasoning capability.

Who created SUPERChem?

SUPERChem was created by Zehua Zhao et al. at Peking University, published in 2025.

SUPERChem — Results

SUPERChem — Evaluations