# Hy4 Preview ![Organization](https://img.shields.io/badge/Org-Hungyuan-6366f1) ![Context](https://img.shields.io/badge/Context-1M%20tokens-1f6feb) ![License](https://img.shields.io/badge/License-Apache%202.0-2da44e) ![Modality](https://img.shields.io/badge/Modality-Text-8957e5) ![Released](https://img.shields.io/badge/Released-Aug%202026-555) > **Quick answer:** Hy4 preview is Tencent Hunyuan's August 2026 flagship — a 770B-parameter Mixture-of-Experts model with 49B active parameters per token, a 1M-token context window, and Apache 2.0 open weights. It scores 92.3% on GPQA Diamond and 82.9% on SWE-bench Multilingual, a major generation-over-generation jump from Hy3. ## At a Glance **Where Hy4 preview leads** - 92.3% GPQA Diamond — near-parity with the strongest closed models - 82.9% SWE-bench Multilingual and 64.3% DeepSWE — large agentic-coding gains over Hy3 (which scored 75.8% and 28.0% respectively) - 1M-token context window with Apache 2.0 open weights — fully self-hostable at frontier scale - Native MTP (multi-token prediction) layer built in for speculative decoding **Where it lags** - Still trails the strongest proprietary models (GPT 5.6 Sol, Claude Opus 5) on several agentic and reasoning benchmarks - Team-acknowledged issues: over-long reasoning traces and a tendency to over-verify its own work - 770B total parameters (49B active) requires substantial multi-GPU infrastructure to self-host at full precision **Best for:** Teams that need a frontier-class, fully open-weight (Apache 2.0) model for agentic coding, tool use, and long-context workloads without API lock-in. ## What Hy4 Preview Is Hy4 preview is the newest flagship in Tencent Hunyuan's "Hy" model line, succeeding Hy3 preview. It's a Mixture-of-Experts model with 78 backbone layers (the first dense, the remaining 77 MoE with 256 routed experts + 1 shared expert, top-8 routed experts active per token), plus a native multi-token-prediction (MTP) layer for speculative decoding. The architecture borrows ideas from DeepSeek and GLM: attention uses Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse, and the residual stream uses identity Hyper-Connections (iHC) to widen inter-layer information flow. Tencent frames this release around agentic productivity rather than raw knowledge benchmarks alone — the team built training data directly from internal software engineers, game developers, finance analysts, and security experts, and reports the largest generation-over-generation gain they've measured, particularly in agentic coding (SWE-bench Multilingual jumped from Hy3's 75.8% to 82.9%; DeepSWE from 28.0% to 64.3%). In a blind internal side-by-side (163 experts rating 203 engineering tasks), Hy4 preview edged out both GLM 5.3 and Kimi K3. This is an early "preview" release — Tencent explicitly flags known issues (over-long reasoning chains, excessive self-verification) and says they expect to iterate quickly, the same pattern that took Hy3 preview from launch to a substantially stronger model. ## Specifications | Field | Value | |---|---| | Organization | Hungyuan (Tencent) | | Architecture | Mixture-of-Experts (MoE), Gated DSA attention | | Total parameters | 770B | | Activated parameters | 49B | | Layers | 78 (1 dense + 77 MoE) | | Routed / shared experts | 256 / 1 (top-8 routed active per token) | | Context length | 1M tokens | | Vocabulary size | 120,832 | | License | Apache 2.0 (open weights) | | Release date | August 2026 | | Modality | Text | ## Pricing Open weights under Apache 2.0 — self-hosted via vLLM or SGLang (official prebuilt images `vllm/vllm-openai:hy4-preview` and `lmsysorg/sglang:hy4-preview`). No API list price; cost is your own inference infrastructure. An FP8-quantized variant (Hy4-preview-FP8) is also published for lower-memory deployment. ## Public Benchmark Scores | Benchmark | Score | Source | Date | |---|---|---|---| | [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) | 92.3% | Tencent Hunyuan model card | 2026-08 | | [Humanity's Last Exam](/benchmarks/center-for-ai-safety-scale-ai/humanitys-last-exam) | 43.4% | Tencent Hunyuan model card | 2026-08 | | [SWE-bench Multilingual](/benchmarks/kabir-khandpur-kilian-lieret-carlos-e-jimenez-ofir-press-john-yang-swe-bench-team/swe-bench-multilingual) | 82.9% | Tencent Hunyuan model card | 2026-08 | | [SWE-bench Pro](/benchmarks/swe-bench-team/swe-bench-pro) | 65.7% | Tencent Hunyuan model card | 2026-08 | | [DeepSWE](/benchmarks/datacurve/deepswe) | 64.3% | Tencent Hunyuan model card | 2026-08 | | [SWE-Marathon](/benchmarks/abundant-ai/swe-marathon) | 31.9% | Tencent Hunyuan model card | 2026-08 | | [Terminal-Bench 2.1](/benchmarks/tbench/terminalbench-21) | 85.4% | Tencent Hunyuan model card | 2026-08 | | [NL2Repo-Bench](/benchmarks/jingzhe-ding-shengda-long-changxin-pu-et-al/nl2repo-bench) | 58.9% | Tencent Hunyuan model card | 2026-08 | | [CyberGym](/benchmarks/zhun-wang-tianneng-shi-et-al-uc-berkeley/cybergym) | 78.4% | Tencent Hunyuan model card | 2026-08 | | [ProgramBench](/benchmarks/vals-ai/programbench) | 17.5% | Tencent Hunyuan model card | 2026-08 | | [WideSearch](/benchmarks/ryan-wong-jiawei-wang-junjie-zhao-li-chen-yan-gao-long-zhang-xuan-zhou-zuo-wang-kai-xiang-et-al-bytedance-seed/widesearch) | 83.9% | Tencent Hunyuan model card | 2026-08 | | [OfficeQA Pro](/benchmarks/databricks/officeqa-pro) | 66.2% | Tencent Hunyuan model card | 2026-08 | | [MCP-Atlas](/benchmarks/scale-ai/mcp-atlas) | 83.7% | Tencent Hunyuan model card | 2026-08 | | [Toolathlon-Verified](/benchmarks/hkust-nlp/toolathlon-verified) | 74.1% | Tencent Hunyuan model card | 2026-08 | | [APEX-Agents](/benchmarks/mercor/apex-agents) | 37.1% | Tencent Hunyuan model card | 2026-08 | | [JobBench](/benchmarks/li-et-al/jobbench) | 61.7% | Tencent Hunyuan model card | 2026-08 | | [GDPval-AA V2](/benchmarks/artificial-analysis/gdpval-aa-v2) | 1678 (Elo) | Tencent Hunyuan model card | 2026-08 | | [CritPt](/benchmarks/zhu-et-al/critpt) | 16.9% | Tencent Hunyuan model card | 2026-08 | | [SWE Atlas — Codebase Q&A](/benchmarks/scale-ai/swe-atlas-codebase-qa) | 64.0% | Tencent Hunyuan model card | 2026-08 | | [SWE Atlas — Test Writing](/benchmarks/scale-ai/swe-atlas-test-writing) | 57.8% | Tencent Hunyuan model card | 2026-08 | | [SWE Atlas — Refactoring](/benchmarks/scale-ai/swe-atlas-refactoring) | 53.3% | Tencent Hunyuan model card | 2026-08 | | [Harbor-Index](/benchmarks/harbor-index-team/harbor-index) | 39.6% | Tencent Hunyuan model card | 2026-08 | | [BankerToolBench](/benchmarks/handshake-ai-research/bankertoolbench) | 78.6% | Tencent Hunyuan model card | 2026-08 | | [SUPERChem](/benchmarks/zehua-zhao-et-al-pku/superchem) | 66.4% | Tencent Hunyuan model card | 2026-08 | | [ArXivMath](/benchmarks/matharena-eth-sri/arxivmath) | 66.6% | Tencent Hunyuan model card | 2026-08 | | [HorizonMath](/benchmarks/erik-y-wang-et-al/horizonmath) | 8.8% | Tencent Hunyuan model card | 2026-08 | | [MathArena Apex 2025](/benchmarks/matharena-eth-sri/matharena-apex-2025) | 74.2% | Tencent Hunyuan model card | 2026-08 | | [BrokenArXiv](/benchmarks/matharena-eth-sri/brokenarxiv) | 54.6% | Tencent Hunyuan model card | 2026-08 | *Scores as self-reported by Tencent Hunyuan for Hy4 preview's own column in the model card's benchmark appendix table (some third-party comparison scores in that same table are Tencent's own re-tests). Several additional benchmarks in the model card (AutomationBench v1.0.6, PostTrainBench V1.1, SkillsBench 79-task text-only subset, Agents' Last Exam ALE-CLI, HLE with-tools variant, and several Tencent-internal-only evals) are not listed here — see [tasks/hy4-preview-progress.md](../../tasks/hy4-preview-progress.md) for why each was excluded (version/subset mismatch with the existing tracked benchmark, or no independent public benchmark page).* ## Hy4 Preview vs Alternatives | Model | GPQA Diamond | SWE-bench Multilingual | License | |---|---|---|---| | **Hy4 Preview** | 92.3% | 82.9% | Apache 2.0 | | [Hy3](/models/hungyuan/hy3) | 90.9% | 75.8% | Proprietary | | [Kimi K3](/models/moonshot-ai/kimi-k3) | 93.5%/92.8%* | 80.8% | Apache 2.0 | | [GLM 5.3](/models/zhipu-ai/glm-5-3) | 91.7%/91.4%* | 81.3% | Proprietary (open weights ~2 weeks post-launch) | | [Qwen3.8 Max](/models/alibaba/qwen3-8-max) | 92.6%/92.2%* | 82.6% | Custom permissive commercial | | [DeepSeek V4 Pro](/models/deepseek/deepseek-v4-pro) | 92.8%/91.7%* | 77.3% | MIT | | [GPT 5.6 Sol](/models/openai/gpt-5-6-sol) | 94.1%/94.7%* | 74.1% | Proprietary | | [Claude Opus 5](/models/anthropic/claude-opus-5) | 93.7%/93.3%* | 89.5%/85.8%* | Proprietary | *Comparison scores marked `*` are Tencent's own re-tests of the other vendors' models (from the same model-card appendix table), not those vendors' own self-reported numbers — see [tasks/hy4-preview-progress.md](../../tasks/hy4-preview-progress.md) for the full sourcing note.* Hy4 preview leads its own generation (Hy3) by a wide margin on both agentic coding and knowledge benchmarks, and is competitive with — though not consistently ahead of — the strongest proprietary models (GPT 5.6 Sol, Claude Opus 5) while being the only model in this set with fully open Apache 2.0 weights at this parameter scale. ## Related Models & Benchmarks - Compare: [Hy3](/models/hungyuan/hy3) · [Kimi K3](/models/moonshot-ai/kimi-k3) · [DeepSeek V4 Pro](/models/deepseek/deepseek-v4-pro) · [GLM 5.3](/models/zhipu-ai/glm-5-3) - Benchmarks: [GPQA Diamond](/benchmarks/rein-et-al/gpqa-diamond) · [SWE-bench Multilingual](/benchmarks/kabir-khandpur-kilian-lieret-carlos-e-jimenez-ofir-press-john-yang-swe-bench-team/swe-bench-multilingual) · [DeepSWE](/benchmarks/datacurve/deepswe) · [Terminal-Bench 2.1](/benchmarks/tbench/terminalbench-21) ## Frequently Asked Questions

What is Hy4 preview?

Hy4 preview is Tencent Hunyuan's August 2026 flagship model — a 770B-parameter (49B active) Mixture-of-Experts model with a 1M-token context window, released under Apache 2.0 open weights.

Is Hy4 preview open source?

Yes — Hy4 preview and an FP8-quantized variant are released under the Apache 2.0 license with weights available on Hugging Face, ModelScope, GitCode, and CNB.

How does Hy4 preview compare to Hy3?

Hy4 preview is a large generation-over-generation jump from Hy3: GPQA Diamond rises from 90.9% to 92.3%, SWE-bench Multilingual from 75.8% to 82.9%, and DeepSWE from 28.0% to 64.3%.

--- *Specs from Tencent Hunyuan's official Hy4-preview Hugging Face model card and GitHub repository (August 2026). Last updated 2026-08-31.*

Hy4 Preview — Training jobs

Training jobs