An Introduction to Retrieval Augmented Generation (RAG)

Source: Dev.to

How RAG Models Work

RAG models consist of two main components:

• Retriever: Retrieves relevant context documents or passages from a large corpus or database given a query or prompt. It ranks and returns the most relevant texts. Popular retrievers include dense retrievers based on bi‑encoders and sparse retrievers based on inverted indexes.

• Generator: Takes the query and retrieved documents as input and generates an output text. The generator is usually a pre‑trained language model like GPT‑3. By conditioning the generation on relevant texts, the generator can produce more factual, specific, and coherent outputs.

During inference, the retriever first fetches relevant contexts for the given query or prompt. The generator then conditions on these contexts, along with the original query, to generate the final output. The retriever and generator are often trained jointly, using a Generative QA (GenQA) objective that maximizes the likelihood of generating the ground‑truth text.

Benefits of RAG Models

• Knowledgeable: By retrieving relevant information, RAG models can incorporate facts and knowledge into generated text, improving factual correctness.
• Specific: Retrieved texts provide useful cues that lead to more focused and specific outputs.
• Coherent: The generator can draw connections between the retrieved contexts and the query to produce holistic, coherent text.
• Scalable training: RAG offers a more sample‑efficient way to inject knowledge into generators compared to training end‑to‑end on massive datasets.

RAG Applications

RAG models have shown promising results on several NLP tasks:

• Question answering: Answer open‑ended questions by retrieving related passages and generating answers from them.
• Summarization: Summarize long documents by retrieving salient passages.
• Dialogue: Enhance chatbots and assistants with more knowledge by retrieving useful context.
• Multi‑hop reasoning: Perform complex inference by chaining and reasoning over multiple retrieved passages.