The one-paragraph version
Retrieval-augmented generation, or RAG, is a way of making a language model answer from real material. When a question comes in, the system first searches a collection, such as the web, a set of documents or a knowledge graph, for relevant passages. It gives those passages to the model along with the question, and the model writes its answer from them. Because the answer is built from retrieved text, the system can show where each part came from. The glossary entry has the short version.
How it works
Three steps, whatever the product:
- Retrieve. Turn the question into a search, and find the passages most likely to contain the answer.
- Augment. Put those passages in front of the model, alongside the question.
- Generate. The model writes an answer using the passages, ideally marking which passage supports which claim.
The quality of the answer depends heavily on step one. A model given the right passage can answer precisely. A model given the wrong one will answer the wrong question well.
Why it was invented
A language model on its own answers from what it absorbed in training, which researchers call parametric memory: knowledge stored in the model's weights. That memory is fixed at training time, hard to update, and impossible to cite.
Lewis et al. introduced retrieval-augmented generation in a paper submitted on 22 May 2020, describing "models which combine pre-trained parametric and non-parametric memory for language generation." The non-parametric memory is the document store, which can be searched and changed without retraining. They found RAG models "generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline."
Specific and factual is exactly what marketing content needs, and exactly what general-purpose AI writing struggles with.
RAG in answer engines
The answer engines your buyers use retrieve from the web when they answer, and their operators document the agents that do the fetching.
- OpenAI says ChatGPT-User is "used for certain user actions in ChatGPT," and that OAI-SearchBot is used "to surface websites in search results in ChatGPT's search features." (OpenAI)
- Perplexity says Perplexity-User may visit a web page "to help provide an accurate answer." (Perplexity)
- Google describes AI Mode "breaking down your question into subtopics and issuing a multitude of queries simultaneously", which is retrieval at scale.
For a publisher, that means three conditions for being used in an answer:
- Retrievable. The engine's crawler must be allowed to reach the page. See the AI crawler audit.
- Liftable. The page needs passages that answer a question on their own, not an answer spread across ten paragraphs of narrative.
- Trustworthy. In the Princeton GEO study, adding quotations, statistics and source citations were among the most effective ways to increase visibility in generated answers.
RAG in content tools
The same design is how grounded AI writing tools work, with one difference that matters: what they retrieve from.
A tool that retrieves from the open web writes from whatever it found, which may be a competitor's description of your product. A tool that retrieves from your own documents writes from your own material, and because every retrieved passage has a known source, every claim can carry a citation back to it. Structuring those documents as a knowledge graph keeps each fact in one place, so drafts stay consistent.
When retrieval goes wrong
RAG is a large improvement over writing from memory, and it is not magic. The failures are predictable:
- The wrong passage is retrieved. A search for a feature pulls a passage about a similarly named one.
- A stale document is retrieved. Last year's pricing sheet is still in the collection.
- Nothing relevant exists. The system either declines, which is correct, or falls back on the model's memory, which is how invention creeps back in.
- The right passage is misstated. The model summarises a correct source incorrectly.
Every one of these is caught faster when each claim shows its source. A reviewer who can click through to the passage sees the mismatch in seconds.
Beyond simple retrieval
Plain retrieval is good at narrow questions answered by one passage and weaker at broad ones whose answer is spread across a collection. Microsoft's GraphRAG research builds a knowledge graph over the collection first, and reports "substantial improvements over a conventional RAG baseline for both the comprehensiveness and diversity of generated answers" for broad questions over large datasets.
What content teams should do
As publishers:
- Let the search and user agents of the engines you care about reach your pages.
- Write passages that answer one question completely, near the top of the section.
- Source every statistic and product claim, and keep time-sensitive pages current.
As buyers of AI writing tools, ask:
- What does the tool retrieve from: the web, a model's memory, or our documents?
- Can every claim in a draft show its source?
- What happens when nothing relevant is retrieved?
- Do we control which documents are used?
- How are outdated documents found and replaced?
- Can we see what was retrieved for a given draft?
How Veritas uses it
Veritas retrieves from a knowledge graph built from the documents you upload, drafts from it, and attaches a numbered citation to every knowledge-based claim. When nothing in your material supports a claim, the claim is left out rather than filled in from the model's memory. That is what grounding means in practice.
Want to see which AI answers mention your brand?Start a 7-day trialSee pricing