Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Summary

Names and formalises retrieval-augmented generation: a generator model conditioned on passages fetched from an external index by a learned retriever. It beat purely parametric models on open-domain question answering and produced more factual text.

Key points

• Splits knowledge into parametric (model weights) and non-parametric (a document index).
• The index can be updated without retraining the model.
• Retrieved passages give a natural way to show provenance.

Why it matters

Origin of the term "RAG" and of the design used, in simplified form, by most document-grounded LLM applications.