Blogerroom logoBlogerroom
AI
AI

Grok 4.5 Ranks Fourth as Hallucination Rate Doubles

AB
Mr. Aayush BhattJuly 14, 20265 min read
๐ŸŒ Language

Grok 4.5 Ranks Fourth as Hallucination Rate Doubles

SpaceXAI's Grok 4.5 ranked fourth on independent benchmarks, not first as Musk claimed, and its hallucination rate doubled to 54%.

"It is an Opus-class model, but faster, more token-efficient and lower cost." Elon Musk posted that on X on July 8, 2026, the day SpaceXAI released Grok 4.5, and he was comparing it directly to Anthropic's flagship line. Independent testers had a different answer within 24 hours. Artificial Analysis, the benchmarking firm most of the industry treats as neutral ground, placed Grok 4.5 fourth on its Intelligence Index, trailing Claude Fable 5, GPT-5.5, and Claude Opus 4.8. Fourth is a genuinely strong result. It is not what "Opus-class" implies.

The launch capped a corporate transition that had been building for months. Two days earlier, on July 6, xAI formally rebranded to SpaceXAI, closing out the integration that began when SpaceX acquired xAI in an all-stock deal back in February 2026. Grok kept its name. Everything else about the business changed.

A Bold Claim Meets an Independent Scoreboard

Artificial Analysis scored Grok 4.5 at 54 on its Intelligence Index v4.1, a composite built from nine demanding evaluations including GDPval-AA v2, Terminal-Bench 2.1, SciCode, Humanity's Last Exam, and GPQA Diamond. That score puts real distance between Grok 4.5 and the top three models on the list, even as it clears the bar for what the industry now calls frontier-class performance. Being fourth among the best models in the world is not a failure by any reasonable measure. It's just not the top spot Musk's post implied, and independent benchmarks exist precisely to catch that gap between marketing language and measured performance.

The Number the Launch Post Left Out

Buried deeper in Artificial Analysis's writeup is the detail that matters most for anyone planning to rely on Grok 4.5 for factual work. On the firm's AA-Omniscience benchmark, which tests both correctness and overconfidence, Grok 4.5's accuracy improved meaningfully, climbing from 35% to 52% compared to the previous version. But its hallucination rate, the rate at which it states wrong answers with confidence rather than acknowledging uncertainty, more than doubled over the same period, from 25% to 54%.

Artificial Analysis frames this as a familiar pattern: larger models tend to know more, but that added knowledge often arrives packaged with more confident wrong answers rather than fewer of them. That tradeoff isn't a footnote. It's a hard constraint on where Grok 4.5 can be trusted unsupervised, and it's the kind of detail that a launch announcement built around beating a well-known rival's naming tier has no incentive to headline.

Why It's Winning Developers Anyway

Despite the accuracy caveat, Grok 4.5 has a real selling point: it's cheap, and it's genuinely competitive on coding. Priced at $2 per million input tokens and $6 per million output tokens, it undercuts Claude Opus 4.8's $5 and $25 rates by more than half, and comes in well below GPT-5.6 Sol's $5 and $30 pricing. On Artificial Analysis's Coding Agent Index, Grok 4.5 scored 76, putting it roughly level with GPT-5.5, while completing benchmark coding tasks for about $2.49 each compared to $5.07 for GPT-5.5 in OpenAI's Codex and $11.80 for Claude Fable 5 running in Claude Code.

Some of that coding strength traces back to Cursor, the developer tool startup SpaceX acquired for $60 billion in June 2026. Cursor contributed trillions of tokens of real developer session data, actual debugging traces, multi-file edits, and error-recovery sequences pulled from production engineering work, to Grok 4.5's training. That's a meaningfully different data source than synthetic coding benchmarks, and it shows up directly in the model's agentic coding results even though it isn't the strongest model on raw intelligence.

The Awkward Discovery at Cursor

That same relationship produced an uncomfortable moment just before launch. Days before Grok 4.5 shipped, Cursor had to pull its own internal benchmark after discovering the model had trained on a snapshot of Cursor's own codebase. In practice, that meant Cursor's benchmark was partly measuring Grok 4.5 against data the model had already seen, which undermines exactly the kind of independent verification Cursor's benchmark was supposed to provide. It's a small, specific example of a much bigger problem the AI industry has been circling for years: as training data pipelines pull from an increasingly tangled web of corporate partnerships and acquisitions, keeping benchmark data genuinely unseen by the model being tested gets harder every generation.

A Shrinking Context Window Nobody Explained

One technical detail stands out precisely because SpaceXAI didn't address it. Grok 4.5 ships with a 500,000-token context window, down from the 1-million-token window in Grok 4.3. The company offered no official explanation for the reduction. Artificial Analysis's plausible inference is that serving a model roughly three times larger in parameter count than its predecessor comes with real infrastructure costs, and shrinking the context window is one lever available to manage that. That's an inference, not a confirmed fact, and the silence around it is notable given how aggressively SpaceXAI has marketed everything else about this release.

Grok 4.5 also isn't available everywhere yet. SpaceXAI cited regulatory compliance review as the reason for delaying EU access, with availability now targeted for mid-July 2026, a gap that keeps a major developer market outside the addressable audience during the model's most closely watched launch window.

The Mandate That Complicates the Whole Picture

There's a separate wrinkle worth noting, one that has nothing to do with benchmarks. Just days after imposing a $200 weekly spending cap on outside AI tools for Tesla's own engineers, Musk directed those same engineers to switch to Grok 4.5, a product built by a company he personally controls. On neutral coding benchmarks, Grok 4.5 still trails Claude Fable 5 by roughly 17 points. That combination, a mandated switch to an internally owned product that performs worse than an outside alternative on measured tests, sits awkwardly next to Musk's own "Opus-class" framing of the launch. The benchmarks tell a story about genuine, if imperfect, progress. The mandate tells a different one about whose incentives are actually driving the adoption numbers.

ShareWhatsAppTwitterLinkedIn
AB

Written by

Mr. Aayush Bhatt

Software Engineer interested in how models work and where they fail.

โ† Back to AI