Model Details
DiffusionGemma 26B-A4B
Quick answer: DiffusionGemma 26B-A4B is Google DeepMind's open-weight diffusion-based language model — a fundamentally different architecture from standard autoregressive LLMs. Rather than generating tokens left-to-right, it iteratively denoises an entire output sequence in parallel, with 26B total parameters and 4B active per denoising step.
At a Glance
Where DiffusionGemma leads
- Novel architecture: generates entire sequences in parallel via iterative denoising, not token-by-token
- Can refine and edit outputs during generation — unlike autoregressive models, intermediate steps are accessible
- 4B active parameters per step gives lower per-step compute than a 26B dense autoregressive model
- Research platform for diffusion LLM capabilities and limitations
Where it lags
- Diffusion LLMs currently trail autoregressive models on most standard benchmarks (MMLU, coding, reasoning)
- Streaming output is fundamentally different — partial results are lower quality until convergence
- Smaller developer ecosystem than standard transformer models
Best for: research on diffusion-based generation, controlled text editing tasks, and applications where parallel refinement is preferable to sequential decoding.
What DiffusionGemma Is
Diffusion language models apply the iterative denoising approach from image diffusion (Stable Diffusion, DALL-E) to text generation. The model starts with a noisy sequence and progressively refines it over multiple forward passes until the output converges to a coherent, high-quality text sequence.
DiffusionGemma 26B-A4B is Google DeepMind's research contribution to this space — using the Gemma 4 architecture (Mixture of Experts, 26B total / 4B active) as the backbone for a diffusion LLM. The open-weight release enables the research community to study, fine-tune, and benchmark diffusion-based language generation.
Specifications
| Field | Value |
|---|---|
| Organization | Google DeepMind |
| Architecture | Diffusion LLM (not autoregressive) |
| Total parameters | 26 billion |
| Active parameters per step | 4 billion |
| License | Gemma Terms of Use |
| Modality | Text |
| Generation method | Iterative denoising |
| HuggingFace | google/diffusiongemma-26b-a4b |
Related Models & Benchmarks
- Compare: Gemma 4 26B-A4B · Gemma 4 31B
- Benchmarks: MMLU-Pro
Frequently Asked Questions
What makes DiffusionGemma different from standard LLMs?
Standard LLMs generate text one token at a time, left to right. DiffusionGemma generates the entire output simultaneously through iterative refinement — starting from noise and converging to a coherent sequence across multiple forward passes.Is DiffusionGemma competitive with autoregressive models?
Not yet at frontier quality levels on standard benchmarks, but it is a rapidly developing research area and represents Google DeepMind's public contribution to open diffusion LLM research.Last updated 2026-06-19.
Pricing
Open weights under Gemma Terms of Use — self-host at no cost.
DiffusionGemma 26B A4B vs Alternatives
| Model | MMLU-Pro | License |
|---|---|---|
| DiffusionGemma 26B A4B | 77.6% | Gemma ToU |
| Gemma 4 26B A4B | — | Gemma ToU |
| Gemma 4 12B | 77.2% | Gemma ToU |
DiffusionGemma 26B A4B uses diffusion decoding (vs autoregressive). Similar MMLU-Pro to Gemma 4 12B (77.6% vs 77.2%).
Scores from Benchgen evaluations. Last updated 2026-07-24.