# MusicCaps ![Category](https://img.shields.io/badge/Category-Multimodal-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Audio%20Quality%20%2B%20Adherence-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-5%2C521%20clips-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-medium-d29922) ![Created](https://img.shields.io/badge/Created-Jan%202023-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2301.11325) > **Quick answer:** MusicCaps is a 5,521-clip dataset of rich, human-expert-written music captions released alongside Google's MusicLM paper, and it has become the de facto standard evaluation set for text-to-music generation systems. ## At a Glance **What it tests:** Whether a text-to-music generation model produces audio that faithfully matches a detailed text description (e.g., "a calming violin melody backed by a distorted guitar riff") while maintaining audio fidelity and coherence over time. **Why it matters:** Before MusicCaps, evaluating text-to-music systems lacked a shared, richly-annotated reference set; its expert-written captions cover both objective musical attributes and subjective qualities, enabling consistent comparison across models like MusicLM, MusicGen, and successors. **Known limitations:** MusicCaps is a fixed 10-second-clip evaluation set, so it primarily measures short-form generation quality rather than long-form musical coherence, and automated metrics (FAD, CLAP score) are still imperfect proxies for genuine musical quality. ## What MusicCaps Measures MusicCaps consists of 5,521 music clips, each paired with a rich text description written by human music experts, covering genre, mood, instrumentation, and other musical attributes in natural language. It was released as supporting material for MusicLM, a hierarchical sequence-to-sequence text-to-music generation model that produces 24kHz audio consistent over several minutes. Because captions are written by experts rather than crowd-workers, MusicCaps descriptions tend to be more musically precise than typical caption datasets, making it a stronger signal for evaluating whether generated audio matches nuanced textual instructions (e.g., matching both instrumentation and mood, not just genre). MusicCaps is most commonly used in two directions: as ground truth for computing similarity-based generation metrics (Fréchet Audio Distance, CLAP score) between generated and reference audio, and as a source of text prompts to drive text-to-music generation for human or automated evaluation. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Multimodal (text-to-music generation evaluation) | | Metric | Fréchet Audio Distance (FAD), CLAP score, and/or Mean Opinion Score (MOS) depending on evaluation setup | | Number of tasks | 5,521 music clips with expert text captions | | Saturation | Medium — generation quality has improved substantially since MusicLM but nuanced adherence remains imperfect | | Created by | Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, and collaborators | | Source paper | [Agostinelli et al. 2023](https://arxiv.org/abs/2301.11325) | | Dataset | [Hugging Face](https://huggingface.co/datasets/google/MusicCaps) | ## How MusicCaps Is Scored Evaluation typically combines an automated audio-similarity metric (Fréchet Audio Distance between generated and reference clips) with a text-audio adherence metric (e.g., CLAP score, measuring how well the generated audio matches the caption), and often human Mean Opinion Score ratings for overall quality and adherence. ## State-of-the-Art Results > Scores vary heavily by generation architecture and evaluation protocol (FAD variant, CLAP model version). See published technical reports for current model-by-model comparisons. ## MusicCaps on Benchgen > No Benchgen results yet — [be the first to run MusicCaps](/benchmarks/agostinelli-et-al-google-research/musiccaps). ## MusicCaps vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **MusicCaps** | Text-to-music generation quality & caption adherence | 5,521 clips | medium | | [MME](/benchmarks/chaoyou-fu-et-al-xiamen-university-tencent-youtu-lab/mme) | Multimodal LLM perception + cognition | 2,374 | high | | [MTVQA](/benchmarks/tang-et-al-bytedance/mtvqa) | Multilingual text-centric visual QA | 6,778 | low | MusicCaps is unique among Benchgen's multimodal benchmarks in evaluating audio generation rather than vision-language understanding, making it a key reference for teams building or fine-tuning music-generation models. ## Run MusicCaps on Your Model Benchgen lets teams evaluate their own text-to-music models against MusicCaps prompts on demand, tracking audio-similarity and caption-adherence metrics over time as generation architectures evolve. ## Related Models & Benchmarks Explore related multimodal benchmarks like [MME](/benchmarks/chaoyou-fu-et-al-xiamen-university-tencent-youtu-lab/mme) and [MTVQA](/benchmarks/tang-et-al-bytedance/mtvqa) to compare vision-language evaluation with MusicCaps' audio-generation focus. ## Frequently Asked Questions

What is MusicCaps?

MusicCaps is a 5,521-clip dataset of expert-written music captions, released alongside Google's MusicLM paper, used as the standard evaluation set for text-to-music generation quality and text adherence.

What does a good MusicCaps score look like?

Because scoring depends on which metric is used (FAD, CLAP score, or MOS) and each has different scales, comparisons are typically made against a specific paper's reported baselines rather than a single universal threshold.

Who created MusicCaps?

MusicCaps was created by Andrea Agostinelli and collaborators at Google Research as part of the MusicLM project.

MusicCaps — Phases

MusicCaps