| Rank | Model | Score |
|---|---|---|
| 1 | ling-3-0-flash-base | 61.1 |
| 2 | ling-3-0-tiny-base | 51.64 |
1 phaseActive
Theorem-driven QA benchmark testing whether models can apply 350+ theorems (math, physics, EE/CS, finance) to solve challenging science problems.
Quick answer: TheoremQA (Chen et al., 2023, EMNLP) is a theorem-driven question-answering dataset of 800 expert-curated questions requiring the application of 350+ theorems across Math, Physics, EE/CS, and Finance. InclusionAI's Ling-3.0-flash-base scores 61.10% accuracy as of 2026.
What it tests: Whether a model can identify and correctly apply the right theorem (e.g. Taylor's theorem, Lagrange's theorem, Huffman coding, Quantum Theorem, Elasticity Theorem) to solve a challenging, domain-specific science or finance problem.
Why it matters: Grade-school-style math benchmarks like GSM8K are largely saturated by frontier models, but applying specific, named theorems to novel problems requires deeper domain knowledge — TheoremQA's original paper found GPT-4 topped out at 51% accuracy with Program-of-Thoughts prompting, and all open-source models of the time stayed below 15%, barely above random guessing.
Known limitations: The dataset spans only 800 questions across a broad span of domains, so per-domain sample sizes are small; scores cited here are self-reported by model developers rather than pulled from a shared third-party leaderboard.
TheoremQA was introduced as the first theorem-driven question-answering dataset, curated by domain experts to evaluate whether AI models can apply theorems — not just recall facts or do arithmetic — to solve challenging problems spanning Math, Physics, Electrical Engineering & Computer Science, and Finance.
| Field | Value |
|---|---|
| Task category | Reasoning (theorem application) |
| Metric | % accuracy |
| Number of questions | 800 |
| Theorems covered | 350+ |
| Domains | Math, Physics, EE/CS, Finance |
| Created by | Chen et al. |
| Org | University of Waterloo |
| Source paper | Chen et al. 2023 (EMNLP) |
| GitHub | TIGER-AI-Lab/TheoremQA |
Models are scored on accuracy — the percentage of the 800 questions answered correctly, typically evaluated with Chain-of-Thought or Program-of-Thought prompting strategies.
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Ling-3.0-flash-base | 61.10% | Ling-3.0-flash-base model card | 2026-11 |
| 2 | Ling-3.0-tiny-base | 51.64% | Ling-3.0-tiny-base model card | 2026-11 |
Scores above are self-reported by InclusionAI on their respective model cards and shown for context. They are not Benchgen measurements, and are not drawn from a shared third-party leaderboard — only the 2 scores above are currently tracked locally.
No Benchgen results yet — be the first to run TheoremQA.
| Benchmark | What it tests | Saturation |
|---|---|---|
| TheoremQA | Theorem-application reasoning across math/physics/EE-CS/finance | Low |
| MMLU-Pro | Broad multi-domain knowledge and reasoning | Medium |
| GSM8K | Grade-school arithmetic word problems | High |
TheoremQA is distinct from arithmetic-style math benchmarks in that it requires identifying and applying a specific named theorem, giving a sharper read on domain-specific reasoning depth rather than general math fluency.
Benchgen lets you evaluate your own model against TheoremQA's theorem-application problem set and track results across model versions.
Benchmark definition paraphrased from Chen et al. 2023. State-of-the-art scores sourced from published model cards and attributed inline. Last updated 2026-11-03.