# Design2Code ![Category](https://img.shields.io/badge/Category-Coding-1f6feb) ![Metric](https://img.shields.io/badge/Metric-Visual%20%2B%20Text%20Similarity-2da44e) ![Tasks](https://img.shields.io/badge/Tasks-484-8957e5) ![Saturation](https://img.shields.io/badge/Saturation-medium-d29922) ![Created](https://img.shields.io/badge/Created-Mar%202024-555) [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b)](https://arxiv.org/abs/2403.03163) [![GitHub](https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white)](https://github.com/NoviScl/Design2Code) > **Quick answer:** Design2Code is the first real-world benchmark for automated front-end engineering, testing whether multimodal LLMs can convert a webpage screenshot directly into working HTML/CSS code that renders back into a visually faithful reproduction of the original design. ## At a Glance **What it tests:** Screenshot-to-code generation across 484 manually curated real-world webpages, scored on fine-grained visual and textual fidelity between the rendered output and the original design. **Why it matters:** Multimodal LLMs converting visual designs directly into code implementations represents a new paradigm for front-end development; Design2Code provides the first rigorous, real-world testbed (rather than synthetic or toy webpages) to measure how close current models are to automating this workflow. **Known limitations:** Automatic metrics (block-match, text, position, color, CLIP similarity) are complemented by human pairwise evaluation because no single automatic metric fully captures perceptual and functional webpage fidelity; the benchmark and its harder Design2Code-HARD variant are also restricted to research-only use. ## What Design2Code Measures Design2Code manually curates 484 diverse real-world webpages as test cases, evaluating how well current multimodal LLMs can generate code implementations that, when rendered, visually match the given reference webpage screenshots. A companion Design2Code-HARD set adds 80 extra-difficult test cases collected from GitHub Pages, specifically chosen to challenge state-of-the-art VLMs' web UI generation capabilities further. The authors develop a suite of automatic evaluation metrics — including block-match, text-content match, spatial-position match, color match, and CLIP-based visual similarity — to assess how faithfully the generated code renders back into the original design, and complement these automatic metrics with comprehensive human pairwise evaluations to validate the resulting performance ranking. Testing frontier models like GPT-4o, GPT-4V, Gemini, and Claude across multiple prompting strategies (direct, text-augmented, and self-revision prompting), the fine-grained metric breakdown showed models mostly struggled with recalling visual elements from the input webpages and generating correct overall layout designs. The authors also release Design2Code-18B, a finetuned open-source model based on CogAgent, along with all model predictions to support further analysis by the research community. ## Benchmark Specifications | Field | Value | |---|---| | Task category | Coding (multimodal screenshot-to-code generation) | | Metric | Block-Match, Text, Position, Color, and CLIP similarity scores, plus human pairwise comparison | | Number of tasks | 484 real-world webpages (plus 80 in the harder Design2Code-HARD set) | | Saturation | Medium — frontier models handle simple layouts reasonably well but struggle with visual element recall and complex layout fidelity | | Created by | Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang | | Source paper | [Si et al. 2024](https://arxiv.org/abs/2403.03163) (NAACL 2025) | | Code | [GitHub](https://github.com/NoviScl/Design2Code) | | Dataset | [Hugging Face](https://huggingface.co/datasets/SALT-NLP/Design2Code) | ## How Design2Code Is Scored Each generated webpage is rendered and automatically compared against the reference screenshot across five metrics (block-match, text, position, color, and CLIP visual similarity), each scored between 0 and 1, then averaged across the test set. Human pairwise evaluation (reference vs. generated) is used to validate that the automatic metrics correlate with genuine human quality judgments. ## State-of-the-Art Results > Scores vary substantially by prompting strategy (direct, text-augmented, self-revision) and model. See the [project page](https://salt-nlp.github.io/Design2Code/) for current standings across GPT-4o, Gemini, Claude, and open-source baselines. ## Design2Code on Benchgen > No Benchgen results yet — [be the first to run Design2Code](/benchmarks/si-et-al-stanford-university/design2code). ## Design2Code vs Other Benchmarks | Benchmark | What it tests | Tasks | Saturation | |---|---|---|---| | **Design2Code** | Screenshot-to-code webpage generation fidelity | 484 | medium | | [MM-Mind2Web](/benchmarks/zheng-et-al-ohio-state-university/mm-mind2web) | Multimodal web-agent action grounding | 2,022 | low | | [MMSearch](/benchmarks/jiang-et-al-cuhk-mmlab/mmsearch) | Multimodal AI search-engine reasoning | 300 | low | Design2Code focuses on generating front-end code from a static design, complementing MM-Mind2Web's focus on taking actions within an existing live webpage. ## Run Design2Code on Your Model Benchgen lets teams run their own multimodal coding models against the Design2Code suite on demand, tracking visual-fidelity metrics (block-match, text, position, color, CLIP similarity) over time to catch regressions in screenshot-to-code generation quality. ## Related Models & Benchmarks Explore related coding and agent benchmarks like [MM-Mind2Web](/benchmarks/zheng-et-al-ohio-state-university/mm-mind2web) and [MMSearch](/benchmarks/jiang-et-al-cuhk-mmlab/mmsearch) to compare front-end code generation with web-agent action-taking and search reasoning. ## Frequently Asked Questions

What is Design2Code?

Design2Code is the first real-world benchmark for automated front-end engineering, testing whether multimodal LLMs can convert 484 webpage screenshots directly into working HTML/CSS code that visually matches the original design.

What does a good Design2Code score look like?

The paper found models mostly lag on recalling visual elements and generating correct layouts, so strong performance shows up as consistently high scores across all five metrics (block-match, text, position, color, CLIP), not just high scores on one dimension like text content.

Who created Design2Code?

Design2Code was created by Chenglei Si and collaborators at the Stanford NLP SALT lab, accepted at NAACL 2025.

Rank	Model	Score
1	gpt-4o	90.4
2	claude-3-opus	87
3	gpt-4v	86.9
4	gemini-pro	84.4

Design2Code — Results

Design2Code — Phases

Design2Code