# FLORES-200 ![Category](https://img.shields.io/badge/Category-Multilingual-1f6feb) ![Metric](https://img.shields.io/badge/Metric-XCOMET--XXL-2da44e) ![Tasks](https://img.shields.io/badge/Directions-1%2C056-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-Medium-d29922) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2207.04672) [![GitHub](https://img.shields.io/badge/GitHub-facebookresearch%2Fflores-181717?logo=github&logoColor=white)](https://github.com/facebookresearch/flores) [![Dataset](https://img.shields.io/badge/Dataset-HuggingFace-ff9d00?logo=huggingface&logoColor=white)](https://huggingface.co/datasets/facebook/flores) > **Quick answer:** FLORES-200 is Meta AI's "No Language Left Behind" evaluation benchmark — a professionally-translated, sentence-aligned parallel corpus spanning 200 languages and 1,056 translation directions. Models are scored with automatic MT-quality metrics; Benchgen tracks XCOMET-XXL, a reference-based neural metric. It's the standard general-translation-quality benchmark for both dedicated MT systems and general-purpose LLMs that report translation capability. ## At a Glance **What it tests:** Sentence-level translation quality across a huge span of language pairs, including many low-resource languages, using a fixed, professionally-translated parallel test set. **Why it matters:** It's the most widely-cited general-purpose MT benchmark in both academic MT research and LLM technical reports — a de facto standard for comparing translation quality across languages a model was never specifically trained to prioritize. **Known limitations:** Sentence-level (not document-level) evaluation, and reported scores vary by which automatic metric is used (XCOMET-XXL vs. CometKiwi vs. BLEU-family metrics) — always confirm which metric a reported number uses before comparing across sources. ## What FLORES-200 Measures FLORES-200 extends Meta AI's original FLORES benchmark to 200 languages, drawn from Wikimedia sources and professionally translated by native speakers, then sentence-aligned across every language in the set. This produces up to 1,056 evaluated translation directions when a model is tested across the full many-to-many matrix (or a curated subset, such as translating into/out of a single pivot language like English or Chinese). Because the source text is identical across every language, FLORES-200 isolates translation quality itself rather than differences in source-text difficulty — making it the standard way MT-specialist models and general LLMs alike report cross-lingual translation capability, including for many lower-resource languages that rarely appear in other multilingual benchmarks. Scoring uses automatic MT-quality metrics rather than exact-match: Benchgen tracks **XCOMET-XXL**, a large reference-based neural evaluation model that correlates strongly with human judgment of translation adequacy and fluency. Papers frequently also report CometKiwi (reference-free) and GEMBA (LLM-judge-based) scores on the same test set — those are documented in prose on individual model pages where reported, but are not separately tracked as leaderboard columns here. ## Benchmark Specifications | Field | Value | |---|---| | Created by | NLLB Team (Meta AI) | | Paper | "No Language Left Behind: Scaling Human-Centered Machine Translation" (arXiv:2207.04672) | | Languages | 200 | | Translation directions | Up to 1,056 (many-to-many) | | Metric (tracked) | XCOMET-XXL, 0-100 scale | | Other metrics reported | CometKiwi (reference-free), GEMBA (LLM-judge), chrF++, spBLEU | | Saturation | Medium — strongest models cluster in the high-80s/low-90s XCOMET-XXL range on average directions | ## How FLORES-200 Is Scored Each translation direction is scored automatically by comparing the model's output against the professional reference translation (for reference-based metrics like XCOMET-XXL) or by evaluating output quality directly (for reference-free metrics like CometKiwi). Scores across evaluated directions are then averaged — commonly reported as an aggregate across all pairs into/out of a pivot language (e.g. "ZH⇔XX", "EN⇔XX") or across the full many-to-many matrix ("XX⇔XX"). Benchgen tracks the XX⇔XX full-matrix average where available, falling back to the broadest reported aggregate. ## State-of-the-Art Results | Rank | Model | Score (XCOMET-XXL) | Source | Date | |---|---|---|---|---| | 1 | Hy-MT2-30B-A3B | 87.47 | Hy-MT2 technical report | 2026-05 | | 2 | Hy-MT2-7B | 86.89 | Hy-MT2 technical report | 2026-05 | | 3 | Hy-MT2-1.8B | 79.77 | Hy-MT2 technical report | 2026-05 | *Scores sourced from the Hy-MT2 technical report's Table 2 (XX⇔XX full-matrix average, XCOMET-XXL column). The paper also reports substantially higher scores for large closed models (e.g. Gemini 3.1 Pro thinking-mode at 92.14) evaluated under the same test set — not yet added to Benchgen pending those models' own page/benchmark-table sync.* ## FLORES-200 on Benchgen > No Benchgen results yet — [be the first to run FLORES-200](/benchmarks/meta-ai/flores-200). ## FLORES-200 vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | FLORES-200 | General multilingual translation quality, 200 languages | 1,056 directions | Medium | | Global-MMLU-Lite | Multilingual knowledge/reasoning (not translation) | — | — | | MGSM | Multilingual grade-school math reasoning | — | — | ## Run FLORES-200 on Your Model Benchgen can evaluate your model's translation quality across FLORES-200's full language matrix with version-controlled, regression-tracked results — catching quality drift across specific language pairs that a single aggregate score would hide. ## Related Models & Benchmarks - Models: [Hy-MT2-30B-A3B](/models/hungyuan/hy-mt2-30b-a3b) · [Hy-MT2-7B](/models/hungyuan/hy-mt2-7b) · [Hy-MT2-1.8B](/models/hungyuan/hy-mt2-1-8b) - Benchmarks: [WMT25](/benchmarks/wmt/wmt25) ## Frequently Asked Questions

What is FLORES-200?

FLORES-200 is Meta AI's benchmark for evaluating machine translation quality across 200 languages, built from professionally-translated, sentence-aligned parallel text spanning up to 1,056 translation directions.

What does a good score look like?

On the XCOMET-XXL metric (0-100 scale), the strongest models score in the high-80s to low-90s on average across directions; scores below 80 typically indicate meaningfully weaker translation quality, especially on lower-resource language pairs.

Who created FLORES-200?

FLORES-200 was created by Meta AI's No Language Left Behind (NLLB) team, described in their 2022 paper "No Language Left Behind: Scaling Human-Centered Machine Translation."

--- *Last updated 2026-08-31.*

FLORES-200 — Results

FLORES-200 — Phases

FLORES-200