# NVIDIA-Nemotron-Labs-Dory1.0-4B ![Organization](https://img.shields.io/badge/Org-NVIDIA-555) ![Context](https://img.shields.io/badge/Context-256K%20tokens-1f6feb) ![Pricing](https://img.shields.io/badge/Price-Open%20weights-2da44e) ![License](https://img.shields.io/badge/License-OpenMDW--1.1-d29922) ![Modality](https://img.shields.io/badge/Modality-Text-8957e5) ![Released](https://img.shields.io/badge/Released-Oct%202026-555) > **Quick answer:** NVIDIA-Nemotron-Labs-Dory1.0-4B is a 4B-parameter looped (latent-recurrent) language model from NVIDIA that applies a shared block of layers repeatedly (1-4 loops) to spend more compute per token, trading inference cost for accuracy from a single checkpoint with no retraining. It supports up to 256K context, was pre-trained natively in NVFP4, and is released under the OpenMDW-1.1 license. ## At a Glance **Where Dory1.0-4B leads** - Elastic inference compute: one checkpoint serves multiple compute budgets by changing loop count (1/2/4) - Memory-efficient looping: last-loop KV caching keeps cache size flat regardless of loop count - Strong IFBench (74.29–79.0 across loop counts) relative to similarly sized peers **Where it lags** - Trails larger models (Gemma4-26B-A4B, Qwen3.6-35B-A3B) on GPQA Diamond and HLE even at 4 loops - Requires unreleased/PR-stage vLLM and llama.cpp support at publication time (not yet in a stable release) **Best for:** general-purpose reasoning on a small footprint where inference-time compute/accuracy tradeoff flexibility matters. ## What Dory1.0-4B Is Dory1.0-4B is NVIDIA's first public "looped" (latent-recurrent) LLM: the layer stack is split into Input, Recurrent, and Output groups, and the shared Recurrent block (20 layers) is applied repeatedly between 8-layer Input and Output blocks, with the input reinjected at every loop. This lets the model spend extra compute on harder tokens without adding parameters or requiring a different checkpoint per compute budget. It was pre-trained on 6T tokens (first ~4.5T in NVFP4, remainder in BF16) plus 300B tokens of context extension to 256K, then went through supervised fine-tuning and multi-environment GRPO reinforcement learning (math, code, science, instruction following, tool use). ## Specifications | Field | Value | |---|---| | Organization | NVIDIA | | Parameters | 4B | | Context window | 256K tokens | | Architecture | Looped (latent-recurrent) normalized-GPT, interleaved sliding-window + full attention | | License | OpenMDW-1.1 | | Release date | 2026-10-16 | | Knowledge cutoff | 2025-12 | | Modality | Text | ## Pricing | | Input (per 1M tokens) | Output (per 1M tokens) | |---|---|---| | Open weights | Free to download | Free to download | Open weights — self-hosting cost only; available in BF16, MXFP8, NVFP4, and Q5_K_M (GGUF) formats. ## Context Window Dory1.0-4B has a **256K-token context window**, extended via a dedicated 300B-token training phase — enough for long documents or extended multi-turn agent sessions in a single request. ## Public Benchmark Scores | Benchmark | Score | Source | Date | |---|---|---|---| | [SWE-bench Verified](/benchmarks/princeton-nlp-openai/swe-bench-verified) | 61.8 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) | 72.16 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [SciCode](/benchmarks/tian-et-al/scicode) | 35.11 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [Humanity's Last Exam](/benchmarks/center-for-ai-safety-scale-ai/humanitys-last-exam) | 10.43 (text-only, no tools, 4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [IFBench](/benchmarks/allenai/ifbench) | 79.0 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [AA-LCR](/benchmarks/artificial-analysis/aa-lcr) | 73 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [AA-Omniscience](/benchmarks/artificial-analysis/aa-omniscience) | 10.03 (accuracy, 4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | | [τ³-bench (Banking)](/benchmarks/sierra-research/3-banking) | 14.69 (4 loops) | [NVIDIA HF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16) | 2026-10 | *Scores above are reported by NVIDIA and shown for context. Figures are measured under NVIDIA's own harness (NeMo Gym / NeMo Evaluator SDK) and are not Benchgen measurements. The 4-loop configuration is used for the headline numbers above; 1-loop and 2-loop results are lower (see model card for the full matrix).* ## Dory1.0-4B vs Alternatives | Model | Context | SWE-bench Verified | GPQA Diamond | Price (in/out per 1M) | |---|---|---|---|---| | **Dory1.0-4B (4 loops)** | 256K | 61.8 | 72.16 | Open weights | | [Qwen3.6-35B-A3B](/models/alibaba/qwen3-6-35b-a3b) | — | 73.4† | 85.35 | Open weights | | [Gemma 4 26B A4B](/models/google/gemma-4-26b-a4b) | — | 57.40 | 80.43 | Open weights | Dory1.0-4B trades parameter count for inference-time compute flexibility — it won't match a 26-35B MoE model on raw reasoning benchmarks, but a single 4B checkpoint can be dialed from cheap (1 loop) to more capable (4 loops) without redeploying. ## How Dory1.0-4B Performs on Real Agent Tasks Terminal-Bench 2.1 and SWE-bench Verified scores improve substantially with more loops (12.5→22.2 and 50.3→61.8 respectively from 1 to 4 loops), suggesting the looped architecture's extra compute budget is especially useful for agentic/tool-use tasks requiring longer reasoning chains, though it still trails much larger dense/MoE models on these same tasks. ## Related Models & Benchmarks - Compare: [Qwen3.6-35B-A3B](/models/alibaba/qwen3-6-35b-a3b) · [Gemma 4 26B A4B](/models/google/gemma-4-26b-a4b) - Benchmarks: [SWE-bench Verified](/benchmarks/princeton-nlp-openai/swe-bench-verified) · [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) · [SciCode](/benchmarks/tian-et-al/scicode) ## Frequently Asked Questions

What is NVIDIA-Nemotron-Labs-Dory1.0-4B?

A 4B-parameter looped (latent-recurrent) language model from NVIDIA that repeatedly applies a shared block of layers to trade inference compute for accuracy from one checkpoint.

What is Dory1.0-4B's context window?

Up to 256K tokens, validated via a dedicated context-extension training phase.

How much does Dory1.0-4B cost?

Open weights — free to download, hosting cost only.

Is Dory1.0-4B open source?

Released under the OpenMDW License Agreement, version 1.1 — open weights for commercial or non-commercial use.

What is Dory1.0-4B's knowledge cutoff?

December 2025.

--- **Excluded vendor benchmarks (bucket C, documented here):** - Terminal-Bench 2.1 — no matching local entity (only Terminal-Bench 3/4/Hard/Science-0.1 exist, a methodology mismatch, not a straight successor). - GDPval-AA-V2 — NVIDIA's reported Elo (815–987 range across loop counts) uses a different, smaller reference-model comparison pool (12 selected models) than convention elsewhere on this benchmark; excluded pending confirmation the harness/reference set matches our existing `gdpval-aa-v2` entity, rather than assumed compatible. *Specs and scores sourced from NVIDIA's official HF model card; third-party benchmark scores attributed inline. Last updated 2026-10-23.*

NVIDIA Nemotron Labs Dory1.0 4B — Training jobs

Training jobs