| Rank | Model | Score |
|---|---|---|
| 1 | ling-3-0-flash-base | 26.9 |
| 2 | ling-3-0-tiny-base | 7.2 |
1 phaseActive
Google DeepMind's curated, de-duplicated, topic-balanced 1,000-prompt refinement of OpenAI's SimpleQA factuality benchmark.
Quick answer: SimpleQA Verified (Haas et al., 2025, Google DeepMind) is a curated, de-duplicated, topic-balanced 1,000-prompt refinement of OpenAI's SimpleQA factuality benchmark. InclusionAI's Ling-3.0-flash-base scores 26.90% correct as of 2026.
What it tests: Whether a model can answer short, fact-seeking questions correctly from parametric knowledge alone, using a cleaned-up, de-duplicated, and topic-balanced subset of the original SimpleQA question pool.
Why it matters: OpenAI's original SimpleQA has known data-quality issues (duplicate and ambiguous questions); SimpleQA Verified was built by Google DeepMind to provide a more reliable short-form factuality signal while remaining comparable in spirit to the original benchmark.
Known limitations: SimpleQA Verified is a distinct, harder-curated dataset from the original simpleqa.md benchmark on Benchgen — scores on the two are not directly comparable. Scores cited here are self-reported by model developers rather than pulled from a shared third-party leaderboard.
SimpleQA Verified evaluates a model's ability to answer short, fact-seeking questions with a single indisputable answer, drawing on a curated, de-duplicated, and topic-balanced 1,000-prompt subset derived from OpenAI's SimpleQA to give a more reliable factuality signal.
| Field | Value |
|---|---|
| Task category | Reasoning (parametric factual knowledge) |
| Metric | % correct |
| Number of questions | 1,000 |
| Created by | Haas et al. |
| Org | Google DeepMind |
| Source paper | Haas et al. 2025 |
| Public leaderboard | Kaggle |
Models are scored on the percentage of the 1,000 curated questions answered correctly, graded against each question's single indisputable answer.
| Rank | Model | Score | Source | Date |
|---|---|---|---|---|
| 1 | Ling-3.0-flash-base | 26.90% | Ling-3.0-flash-base model card | 2026-11 |
| 2 | Ling-3.0-tiny-base | 7.20% | Ling-3.0-tiny-base model card | 2026-11 |
Scores above are self-reported by InclusionAI on their respective model cards and shown for context. They are not Benchgen measurements, and are not drawn from a shared third-party leaderboard — only the 2 scores above are currently tracked locally.
No Benchgen results yet — be the first to run SimpleQA Verified.
| Benchmark | What it tests | Saturation |
|---|---|---|
| SimpleQA Verified | Curated, de-duplicated short-form factuality | Low |
| SimpleQA | Original short-form factuality (OpenAI) | Medium |
SimpleQA Verified is distinct from the original SimpleQA in that its question pool has been de-duplicated and topic-balanced by Google DeepMind, making it a stricter, non-interchangeable factuality signal rather than a drop-in replacement.
Benchgen lets you evaluate your own model against SimpleQA Verified's curated factuality question set and track results across model versions.
Benchmark definition paraphrased from Haas et al. 2025. State-of-the-art scores sourced from published model cards and attributed inline. Last updated 2026-11-03.