What's the actual hallucination rate of ChatGPT or Claude in 2026?

On Vectara's short-document leaderboard (HHEM): GPT-4o is around 1.5%, Claude Sonnet 4.4%, Claude Opus 10.1%. On Vectara's harder enterprise benchmark covering 7,700+ articles across law, medicine, finance, and technology, reasoning models including GPT-5 and Claude Sonnet 4.5 exceed 10%. Real-world rates vary widely with prompt complexity, domain specialization, and whether retrieval is used.

Does RAG (retrieval-augmented generation) eliminate hallucinations?

No. RAG reduces hallucinations meaningfully (the 2025 hybrid-architecture survey reports 35-60% error reduction over baseline LLMs), but Stanford's research on purpose-built legal AI tools showed RAG-based products still hallucinate 17% of the time. The standard-of-care today combines RAG with span-level verification: each generated claim is matched against retrieved evidence and flagged if unsupported.

Why do reasoning models (GPT-5, Claude Sonnet 4.5) hallucinate more on summarization?

Reasoning models invest computational effort in chain-of-thought processing, which leads them to add inferences, draw connections, and generate insights that go beyond the source document. For grounded summarization tasks (where staying inside the source is the goal), this is a regression. The Vectara updated leaderboard documents this pattern across multiple reasoning models.

How would I measure hallucination rate in my own marketing content?

The standard methodology mirrors Vectara's: have the AI summarize a known source document, then check each generated claim against the source. Score each claim 0 (clearly supported), 0.5 (ambiguously supported), or 1 (unsupported or contradicted). Run this on a sample of 50-100 generated outputs. The percentage of claims scoring 1 is your effective hallucination rate.

Can I trust AI-generated stats in a marketing post?

Only if every quantitative claim links to a primary source you've personally verified. The published hallucination rates apply specifically to numbers, dates, and citations: AI tools fabricate these at higher rates than other content types, because they're patterns the model has been trained to produce convincingly. Treat any unsourced statistic in AI output as suspect by default.

Will hallucination rates keep dropping?

Probably yes for short, well-defined contexts (Vectara's leaderboard has shown steady improvement on the easier benchmark). Probably no for harder enterprise benchmarks, where reasoning models have actually regressed. The trend in 2026 is that hallucinations are not a model-scaling problem you can wait out; they require structural interventions (RAG, verification, citation) regardless of model improvements.

What the Research Actually Says About AI Hallucinations in Marketing Content

Why this question matters for marketers

There's a popular framing in 2026 that "AI hallucinations are mostly a solved problem now." It's not true. The published evidence shows that hallucination rates have improved meaningfully on simple benchmarks, plateaued or regressed on harder ones, and remain a structural problem that no amount of model scaling has fully eliminated.

For marketing teams, the question isn't academic. AI-generated marketing content with fabricated statistics, invented product capabilities, or misattributed quotes is not a hypothetical risk. It's a daily occurrence in workflows that don't have grounding and verification built in. This guide synthesizes what the published research actually says, separated from vendor marketing claims about hallucination, so you can calibrate your team's AI use based on evidence rather than vibes.

The benchmark landscape

Three independent benchmarking efforts have produced the most-cited public data on hallucination rates.

Vectara's Hallucination Evaluation Leaderboard (HHEM)

Vectara's hallucination leaderboard is the most widely cited public benchmark. Its methodology: have an LLM summarize a short document, then score each generated claim against the source. The HHEM score ranges from 0 to 1, where any value below 0.5 counts as a hallucination.

On the original short-document benchmark, the published 2025 results put frontier models at:

Google Gemini-2.0-Flash: 0.7% hallucination rate
OpenAI GPT-4o: 1.5%
Anthropic Claude Sonnet (3.x): 4.4%
Anthropic Claude Opus (3.x): 10.1%

Vectara then updated the leaderboard with a much harder benchmark covering 7,700+ articles spanning law, medicine, finance, education, and technology. On this enterprise-relevant dataset:

Gemini-2.5-flash-lite leads with a 3.3% hallucination rate
Reasoning models including GPT-5, Claude Sonnet 4.5, and Grok-4 all exceed 10% hallucination rates

The headline finding from this update: reasoning models perform worse on grounded summarization than non-reasoning models. They invest computational effort in chain-of-thought processing, which leads them to add inferences, draw connections, and generate insights that go beyond the source document. For tasks where staying inside the source is the goal, the "reasoning" capability becomes a regression.

Galileo Labs' Hallucination Index

Galileo's Hallucination Index takes a different methodology: it ranks 22 leading LLMs across context lengths from 1,000 to 100,000 tokens, focused specifically on retrieval-augmented generation (RAG) workflows. The Index uses two metrics, Correctness and Context Adherence, evaluated with Galileo's ChainPoll method.

Key findings from the Galileo Index:

Anthropic's Claude 3.5 Sonnet was evaluated as the most accurate LLM overall.
Google's Gemini 1.5 Flash ranked as the best value-for-money model.
Open-source models including Gemma, Llama, and Qwen continue to close the gap on closed-source leaders, especially at longer context lengths.

The most relevant takeaway for marketers: closed-source frontier models still lead on accuracy, but the lead is shrinking, and pricing-adjusted, smaller models often produce comparable results for content tasks.

The Stanford HAI legal AI study

The most rigorous domain-specific hallucination study published is Stanford's research on legal AI tools, conducted by RegLab and HAI (summary here, full paper in Journal of Legal Analysis).

What the study found:

General-purpose LLMs hallucinate on 69% to 88% of specific legal queries, with hallucination rates exceeding 75% when asked about a court's core ruling.
Even purpose-built legal AI tools (LexisNexis's Lexis+ AI, Thomson Reuters' Westlaw AI-Assisted Research, Ask Practical Law AI) hallucinate 17% of the time.
These tools use retrieval-augmented generation (RAG) and were marketed as "hallucination-free." The Stanford study showed that claim was substantially overstated.

The Stanford team also drew a critical distinction relevant to citation-first AI: a hallucination is content that's factually wrong (the AI invents a fact). A misgrounded citation is content that's factually right but cites a source that doesn't actually support the claim. Both occur at meaningful rates in legal AI. For marketing content the practical implication is the same: every cited claim should be verified at the source, not just at the citation.

What research says about why LLMs hallucinate

The 2025 academic literature converges on a few causal mechanisms (recent survey paper at arxiv.org/abs/2510.24476):

Knowledge-based hallucinations arise when the model lacks reliable training data on the topic and fills the gap with plausible-sounding fabrication. These are the "made-up statistics" failures most marketers worry about.

Logic-based hallucinations arise when the model has correct underlying knowledge but generates inferences that don't follow from it. These are subtler: the source data is correct, but the reasoning chain that produces the output adds claims not supported by it.

Inference-time sampling failures occur when the model's randomness (temperature, top-p sampling) selects a low-probability completion that conflicts with what the model "knows" with higher confidence.

The literature also identifies architectural biases: certain transformer behaviors (like the tendency to generate fluent text regardless of factual support) are baked into the model architecture and cannot be removed without rebuilding the architecture itself.

What works as mitigation

The 2025 research is unusually consistent on which interventions reduce hallucinations measurably. Three approaches have evidence behind them:

1. Retrieval-augmented generation (RAG). The dominant mitigation in production today. The 2025 hybrid-architecture survey reports a 35-60% error reduction over baseline LLMs across multiple benchmarks. RAG works by retrieving relevant source documents at query time and providing them to the model as context, constraining what the model can plausibly generate.

The catch: RAG itself is not a complete fix. Stanford's legal AI study showed that purpose-built RAG-based tools still hallucinate at 17% rates. RAG reduces hallucinations; it does not eliminate them.

2. Span-level verification. A newer intervention with strong recent evidence. Each generated claim is matched against the retrieved source documents and flagged if unsupported. Best-in-class systems combine RAG with automatic span checks and surface those verifications to the user. This is the architecture pattern emerging as state-of-the-art for high-stakes domains.

3. Mandatory citation. Forcing the model to cite the specific source for every factual claim doesn't fix hallucination directly, but it makes hallucination detectable: the user can verify each claim against its citation. The Princeton GEO paper (Aggarwal et al., 2023) measured a 41% visibility lift in generative-engine answers when content has embedded statistics with citations, plus 28% for quotations and 115% for sourced citations on lower-ranked content. The signal is that citation-rich content is treated differently by both upstream training and downstream retrieval.

What does not work as mitigation, based on published evidence:

Larger models alone. Vectara's leaderboard shows that newer, larger reasoning models often perform worse on grounded tasks than older, smaller models.
Prompt engineering ("be accurate, don't make things up"). Multiple studies have shown this produces zero measurable reduction in hallucination rates.
Higher inference temperatures with self-consistency checks. Some early papers proposed this; subsequent work showed the gains were marginal and inconsistent.

What this means for marketing content specifically

There's relatively little marketing-specific hallucination research published, but the cross-domain pattern is consistent enough to draw confident conclusions.

Statistics and dates are the highest-risk category. AI tools fabricate these at higher rates than other content types because they're patterns the model has been trained to produce convincingly. Vectara's findings on enterprise content (10%+ hallucination on harder benchmarks) and Stanford's findings on legal queries (17%+) both point toward a real risk that any specific number, percentage, date, or proper noun in AI-generated marketing content carries non-trivial probability of being fabricated.

Product claims are the second-highest risk. When a marketer prompts a generic AI writer to "write a blog about how our product solves X," the model fills in product capabilities from pattern-matching to similar products in its training data. The result is fluent, plausible, and frequently wrong about what your product actually does.

Brand voice is unaffected by hallucination but vulnerable to drift. Hallucination is a factual-content problem, not a stylistic one. AI tools can mimic brand voice convincingly while hallucinating facts. This is the "AI-written, AI-discounted" pattern: the prose reads correctly but the underlying claims aren't grounded.

Competitive comparisons are the most sensitive category for litigation risk. Hallucinated claims about competitors (capabilities they don't have, pricing they don't charge, customers they don't serve) can create defamation exposure. The Stanford legal AI work makes clear that purpose-built RAG systems still hallucinate at material rates. Generic AI writers fare worse.

The practical implication

For marketing teams generating content with AI in 2026, the research supports a specific operating posture:

Assume hallucination rates of 5-15% on factual content even with RAG-grounded tools. The evidence does not support claims that any commercial AI writer produces hallucination-free output.
Treat every quantitative claim as suspect until verified at primary source. Hallucination concentrates in numbers and dates.
Use citation-first tools that source every claim and flag unsupported ones, rather than tools that produce fluent prose without traceable grounding.
Run sample audits. Periodically check 30-50 AI-generated outputs against primary sources to measure your actual hallucination rate. The published benchmarks are reference points, not predictions of your specific tool's behavior.
Avoid reasoning models for grounded summarization. Vectara's data is consistent: reasoning models are worse at staying inside the source. Use them for analysis, not for content extraction.

These aren't conservative defaults; they're what the published research supports. Anyone selling you a marketing AI tool that promises "no hallucinations" is selling something the entire research literature says doesn't yet exist.

Closing

Hallucination is the central reliability problem of contemporary LLMs. The research consensus is that it's reducible but not eliminable through model improvements alone, that structural interventions (RAG, verification, citation) carry the actual evidence, and that even state-of-the-art purpose-built tools still hallucinate at rates marketers should care about.

For B2B marketing teams, the implication is that the tools you choose should be measured against this evidence rather than against vendor claims. A tool that grounds every claim in a structured knowledge source and surfaces span-level verifications to the editor is materially different from one that produces fluent prose without traceable sourcing. The first carries the architectural patterns the research supports. The second is a polished version of the problem the research describes.

Veritas builds the architectural pattern this article describes: knowledge-graph-grounded content generation with mandatory citation on every claim and span-level verification before publish. Try Veritas free or explore Content Generation.