Grok 4

Grok 4

Quick answer: Grok 4 is xAI's frontier reasoning model, released 9 July 2025. It was trained with large-scale reinforcement learning on xAI's Colossus cluster and ships with native tool use and real-time search across X and the web, plus a 256K-token context window. xAI reports 87.5% on GPQA, 91.7% on AIME 2025, and 38.6% on Humanity's Last Exam (with tools). It is a closed, API-only model.

At a Glance

Where Grok 4 leads

• Frontier math and science reasoning — 87.5% on GPQA, 91.7% on AIME 2025, 90% on HMMT 2025.
• Native, RL-trained tool use with real-time search across X and the web.
• Strong long-horizon planning — tops Vending-Bench with $4,694.15 net worth (avg of 5 runs).

Where it lags

• A 256K context window, smaller than the 1M windows of Claude Opus 4.6 and Gemini 3 Pro.
• Many headline scores rely on tools or the Grok 4 Heavy variant; closed and API-only with no open weights.

Best for: tool-driven, search-heavy reasoning and math workloads.

What Grok 4 Is

Grok 4 is xAI's flagship model, built by scaling reinforcement learning to pretraining levels on the 200,000-GPU Colossus cluster. xAI describes it as setting a new state of the art for closed models on reasoning, and a defining feature is native tool use: Grok 4 was trained with RL to drive a code interpreter, browse the web, and run advanced keyword and semantic search across X — choosing its own queries and digging as deep as a research question requires. A more powerful parallel-test-time-compute variant, Grok 4 Heavy, considers multiple hypotheses at once and was the first model to cross 50% on Humanity's Last Exam.

Its benchmark profile is heavily weighted toward math, science, and reasoning: 87.5% on GPQA, 91.7% on AIME 2025, 90% on HMMT 2025, and a frontier-leading 15.9% on the deliberately hard ARC-AGI-2. On the agentic Vending-Bench business simulation, xAI reports Grok 4 finishing with $4,694.15 net worth (averaged across five runs), well ahead of other models and human baselines — a long-horizon planning signal. The API exposes a 256K-token context window and multimodal understanding across text and vision.

For Benchgen, Grok 4 is an API-benchmarked model with no open weights. xAI's published scores show strong reasoning and search behaviour, but only a verified, repeated-trial run on a real domain task reveals whether that translates into reliable end-to-end completion under a consistent rubric. That reliability-under-repetition view is what Benchgen is built to surface.

Specifications

Pricing

Prompts above 128k tokens are billed at $6.00 / $30.00 per 1M. Source: xAI API pricing.

Context Window

Grok 4 has a 256K-token context window — roughly 380 pages of text in a single request. That is smaller than the 1M-token windows of Claude Opus 4.6 and Gemini 3 Pro, but ample for large documents and multi-file codebases, and it pairs with native tool use and real-time search to pull in fresh context beyond the window.

Public Benchmark Scores

Scores above are reported by xAI and shown for context. They are not Benchgen measurements. Many figures use tools (Python interpreter and internet search), and the Grok 4 Heavy variant posts higher numbers (e.g. 50.7% on Humanity's Last Exam text-only subset, 100% on AIME 2025 with Python). See the source for methodology.

Grok 4 vs Alternatives

Grok 4's reasoning is frontier-level and its native tool use plus real-time X/web search are genuine differentiators, but its 256K context window trails rivals' 1M windows and several headline scores depend on tools or the Heavy variant. For a builder, Grok 4 is the pick when search-grounded, tool-driven reasoning matters more than raw context size. (Rival scores attributed to OpenAI and Google; not Benchgen measurements.)

How Grok 4 Performs on Real Agent Tasks

Grok 4's most agent-relevant public result is Vending-Bench: xAI reports it dominating the simulated business with $4,694.15 net worth and 4,569 units sold (averaged over five runs), far ahead of Claude Opus 4 and human baselines. Combined with native, RL-trained tool use — autonomous web and X search, code execution — Grok 4 is clearly built for multi-step, tool-driven workflows rather than single-turn answers.

What those benchmarks cannot tell you is how Grok 4 behaves on a specific domain workflow, run many times, under a consistent rubric. Vending-Bench measures planning in a synthetic retail sim; what agent builders actually need is reliable, repeated completion of the task in front of them — including how often the model fails, and why. That reliability-under-repetition view is what Benchgen is built to surface, and it is the dimension public benchmarks leave open.

Use Grok 4 via API

from openai import OpenAI  # xAI exposes an OpenAI-compatible API

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.x.ai/v1")

response = client.chat.completions.create(
    model="grok-4",
    messages=[{"role": "user", "content": "Summarize this contract clause..."}],
)
print(response.choices[0].message.content)

Related Models & Benchmarks

• Compare: GPT-5 · Claude Opus 4.6 · Gemini 3 Pro · Claude Sonnet 4.6
• Benchmarks: GPQA Diamond · AIME 2025

Frequently Asked Questions

Specs and scores sourced from xAI's Grok 4 announcement. Third-party benchmark scores attributed inline to xAI. Benchgen scores, where shown, are produced on Benchgen infrastructure. Last updated 2026-06-17.