FLORES-200

FLORES-200

Quick answer: FLORES-200 is Meta AI's "No Language Left Behind" evaluation benchmark — a professionally-translated, sentence-aligned parallel corpus spanning 200 languages and 1,056 translation directions. Models are scored with automatic MT-quality metrics; Benchgen tracks XCOMET-XXL, a reference-based neural metric. It's the standard general-translation-quality benchmark for both dedicated MT systems and general-purpose LLMs that report translation capability.

At a Glance

What it tests: Sentence-level translation quality across a huge span of language pairs, including many low-resource languages, using a fixed, professionally-translated parallel test set. Why it matters: It's the most widely-cited general-purpose MT benchmark in both academic MT research and LLM technical reports — a de facto standard for comparing translation quality across languages a model was never specifically trained to prioritize. Known limitations: Sentence-level (not document-level) evaluation, and reported scores vary by which automatic metric is used (XCOMET-XXL vs. CometKiwi vs. BLEU-family metrics) — always confirm which metric a reported number uses before comparing across sources.

What FLORES-200 Measures

FLORES-200 extends Meta AI's original FLORES benchmark to 200 languages, drawn from Wikimedia sources and professionally translated by native speakers, then sentence-aligned across every language in the set. This produces up to 1,056 evaluated translation directions when a model is tested across the full many-to-many matrix (or a curated subset, such as translating into/out of a single pivot language like English or Chinese).

Because the source text is identical across every language, FLORES-200 isolates translation quality itself rather than differences in source-text difficulty — making it the standard way MT-specialist models and general LLMs alike report cross-lingual translation capability, including for many lower-resource languages that rarely appear in other multilingual benchmarks.

Scoring uses automatic MT-quality metrics rather than exact-match: Benchgen tracks XCOMET-XXL, a large reference-based neural evaluation model that correlates strongly with human judgment of translation adequacy and fluency. Papers frequently also report CometKiwi (reference-free) and GEMBA (LLM-judge-based) scores on the same test set — those are documented in prose on individual model pages where reported, but are not separately tracked as leaderboard columns here.

Benchmark Specifications

How FLORES-200 Is Scored

Each translation direction is scored automatically by comparing the model's output against the professional reference translation (for reference-based metrics like XCOMET-XXL) or by evaluating output quality directly (for reference-free metrics like CometKiwi). Scores across evaluated directions are then averaged — commonly reported as an aggregate across all pairs into/out of a pivot language (e.g. "ZH⇔XX", "EN⇔XX") or across the full many-to-many matrix ("XX⇔XX"). Benchgen tracks the XX⇔XX full-matrix average where available, falling back to the broadest reported aggregate.

State-of-the-Art Results

Scores sourced from the Hy-MT2 technical report's Table 2 (XX⇔XX full-matrix average, XCOMET-XXL column). The paper also reports substantially higher scores for large closed models (e.g. Gemini 3.1 Pro thinking-mode at 92.14) evaluated under the same test set — not yet added to Benchgen pending those models' own page/benchmark-table sync.

FLORES-200 on Benchgen

No Benchgen results yet — be the first to run FLORES-200.

FLORES-200 vs Other Benchmarks

Run FLORES-200 on Your Model

Benchgen can evaluate your model's translation quality across FLORES-200's full language matrix with version-controlled, regression-tracked results — catching quality drift across specific language pairs that a single aggregate score would hide.

Related Models & Benchmarks

• Models: Hy-MT2-30B-A3B · Hy-MT2-7B · Hy-MT2-1.8B
• Benchmarks: WMT25

Frequently Asked Questions

Last updated 2026-08-31.