Who invented retrieval-augmented generation?

The term comes from a 2020 paper by Patrick Lewis and eleven co-authors, submitted to arXiv on 22 May 2020, which combined a pre-trained sequence-to-sequence model with a neural retriever over a dense vector index of Wikipedia.

Is ChatGPT search a RAG system?

In the general sense, yes: when ChatGPT searches the web, it retrieves pages and writes an answer from them with citations. OpenAI uses OAI-SearchBot to surface sites in those search features and ChatGPT-User to fetch pages a user asks about.

Does RAG stop hallucinations?

It reduces them but does not remove them. The original paper found RAG produced more specific and factual text than a model without retrieval, but a model can still misread a source, mix two sources, or fill gaps when retrieval returns nothing useful.

How do I get my content retrieved by AI engines?

Let their search crawlers in, make sure pages are indexed and eligible for snippets, put key facts in plain HTML text, and answer specific questions on specific pages so your page matches the sub-queries the engine runs.

AI search glossary

Retrieval-augmented generation (RAG): how AI answers find their sources

Retrieval-augmented generation (RAG) is a method in which a language model first retrieves relevant documents from an external source, such as a search index or a document database, and then generates its answer using those documents as context instead of relying only on what it memorized in training.

Also called: RAG, retrieval augmented generation, grounded generation

Why it matters for AI search

Every AI search answer that cites links is a form of RAG: the engine retrieves pages, then writes from them. That makes the retrieval step the gate. If your page is not crawlable, not indexed, or does not match the sub-questions the engine searches for, it never enters the context window, however good it is. Being retrievable is the first requirement for being recommended.

Where RAG comes from

Lewis and colleagues introduced the term in a 2020 paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks". Their starting point: large language models store facts in their weights (parametric memory) but struggle to access and update that knowledge precisely. They paired a seq2seq model with a non-parametric memory, a dense vector index of Wikipedia searched by a neural retriever. They tested two versions, one that uses the same retrieved passages for the whole answer and one that can use different passages per token. The approach set state-of-the-art results on three open-domain question answering tasks and produced more specific, diverse and factual text than the model alone.

How a RAG pipeline works

Query. The user's question is turned into one or more search queries. AI search engines often split it into many sub-queries, a step Google calls query fan-out.
Retrieve. The system pulls candidate documents from an index: a web search index, a vector database of your company's documents, or both.
Select. The best passages are ranked and trimmed to fit the model's context window.
Generate. The model writes an answer conditioned on those passages, often citing them.

The same pattern runs in a customer support bot over a help center and in Google AI Mode over the open web. Only the index differs.

What RAG means for AI visibility

For public AI search, the retrieval index is a search engine or a crawler-built index. Google says a page must be indexed and eligible to show a snippet to appear as a supporting link in AI Overviews or AI Mode. OpenAI says blocking OAI-SearchBot keeps a site out of ChatGPT search answers. Practical steps:

Be retrievable. Allow the search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot). You can check your rules with the robots.txt checker.
Be readable without scripts. Many retrieval crawlers read raw HTML.
Be passage-friendly. Retrieval works on chunks. A paragraph that states the fact and its context on its own is easier to select than one that depends on the previous three.
Be specific. Pages that answer one sub-question plainly match more of the fan-out queries.

RAG lowers but does not remove hallucination, and it is the base on which agentic search adds planning and multi-step browsing.

Questions

Who invented retrieval-augmented generation?: The term comes from a 2020 paper by Patrick Lewis and eleven co-authors, submitted to arXiv on 22 May 2020, which combined a pre-trained sequence-to-sequence model with a neural retriever over a dense vector index of Wikipedia.
Is ChatGPT search a RAG system?: In the general sense, yes: when ChatGPT searches the web, it retrieves pages and writes an answer from them with citations. OpenAI uses OAI-SearchBot to surface sites in those search features and ChatGPT-User to fetch pages a user asks about.
Does RAG stop hallucinations?: It reduces them but does not remove them. The original paper found RAG produced more specific and factual text than a model without retrieval, but a model can still misread a source, mix two sources, or fill gaps when retrieval returns nothing useful.
How do I get my content retrieved by AI engines?: Let their search crawlers in, make sure pages are indexed and eligible for snippets, put key facts in plain HTML text, and answer specific questions on specific pages so your page matches the sub-queries the engine runs.