AI Foundations · Chapter 7
What is RAG?
Understand Retrieval-Augmented Generation and how it helps AI systems answer using documents, databases, and external knowledge.
Introduction
RAG stands for Retrieval-Augmented Generation. It is a common technique used to make AI systems answer questions using external information.
Instead of relying only on what a model learned during training, a RAG system first retrieves relevant information from documents, databases, websites, or knowledge bases.
Why RAG is Needed
Large Language Models are powerful, but they do not automatically know your private company documents, latest policies, internal systems, or constantly changing information.
RAG helps solve this by connecting the AI model to external knowledge.
Simple RAG Flow
- User asks a question
- The system searches relevant documents
- The most useful content is retrieved
- The retrieved content is sent to the LLM
- The LLM generates an answer using that context
A Simple Example
Imagine an employee asks: “What is our company travel reimbursement policy?”
A normal AI model may guess or give a generic answer. A RAG system can search the actual company policy document and answer based on that specific source.
Where RAG is Used
- Company knowledge assistants
- Document Q&A systems
- Customer support bots
- Legal document search
- Healthcare knowledge systems
- Technical documentation assistants
- Enterprise AI copilots
RAG vs Fine-Tuning
RAG is usually better when information changes often or when the AI needs access to private documents.
Fine-tuning changes model behavior by training it further. RAG gives the model fresh context without retraining it.
Common RAG Components
- Documents: the source knowledge.
- Embeddings: numerical representations of meaning.
- Vector database: stores and searches embeddings.
- Retriever: finds relevant content.
- LLM: generates the final answer.
Common Mistakes
- Adding too many irrelevant documents
- Using poor document chunking
- Not checking answer quality
- Ignoring source citations
- Assuming RAG removes all hallucinations
Summary
RAG is one of the most important patterns in practical AI. It allows AI systems to answer using external knowledge instead of relying only on model training.
In the next chapter, we will understand tokens and why they matter for AI cost, context, and response quality.