Retrieval-augmented generation (RAG)

A plain language model answers from what it absorbed in training. A RAG system adds a step in front of it. The question is used to search an index, which might be the open web for an answer engine or a company’s own documents for an internal tool, and the best-matching passages are placed in the model’s context. The model then writes its answer from those passages and can point back to them.

The retrieval step is usually semantic rather than keyword-based: documents are split into chunks, each chunk is turned into a vector that captures its meaning, and the chunks closest in meaning to the question are returned. That is why a RAG system can find a passage that answers a question without sharing its exact words.

For AI search, RAG is why a page can be cited the week it is published. Perplexity, ChatGPT search and Google’s AI features fetch pages at answer time, so the engine does not need to have learned about you in training. It needs to find you, read you and judge you worth quoting.

For content generation, RAG is the difference between a model guessing what your company does and a model reading your own material. The original research framed exactly this gap: models store knowledge in their parameters but struggle to show where a fact came from or to update it, and retrieval addresses both.

• The paper that named the method describes models “which combine pre-trained parametric and non-parametric memory for language generation”, with the non-parametric memory being “a dense vector index of Wikipedia”.Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, May 2020), 15 Sept 2026
• It identifies the problem retrieval addresses: “providing provenance for their decisions and updating their world knowledge remain open research problems.”Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, May 2020), 15 Sept 2026
• Its authors report that RAG models “generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.”Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, May 2020), 15 Sept 2026
• Vectara’s hallucination leaderboard notes that models are “increasingly used in RAG (Retrieval Augmented Generation) and Agentic pipelines”, where the model acts as “a summarizer of the search results”.Vectara, Hallucination Leaderboard, 15 Sept 2026

1. 1

   For your own site: make key pages fetchable without JavaScript, put the answer to the page’s main question in its opening paragraph, and keep one fact in one place so retrieval cannot surface two versions of it.

2. 2

   Write sections that stand on their own. Retrieval returns chunks, not whole pages, so a paragraph that only makes sense after the previous three is a weak chunk.

3. 3

   For a content tool: build the index from approved, current documents only, and delete superseded versions rather than leaving them to compete.

4. 4

   Ask the vendor how chunks are cited. The useful answer is a citation per claim that opens the passage, not a list of files consulted.

5. 5

   Test retrieval directly with questions your buyers ask, and check which passages come back before judging the written answer.

• Assuming RAG makes output accurate. It makes output traceable. The model can still misread a passage, so citations still need opening.
• Indexing everything. Old pricing, draft decks and competitor notes all become retrievable facts once they are in the index.
• Judging a RAG tool by how fluent it sounds. Fluency comes from the model; accuracy comes from what was retrieved.

Grounding

Grounding is the goal: output traceable to sources. RAG is the most common way to get there.

Fine-tuning

Fine-tuning changes what a model has learned and needs retraining to update. RAG leaves the model alone and changes what it reads, so updating a document updates the answer.

1. 1Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, May 2020) read 15 Sept 2026
2. 2Vectara, Hallucination Leaderboard read 15 Sept 2026