An LLM doesn't know your company's docs, last week's news, or your private data — and it'll confidently make things up. RAG fixes that: retrieve the relevant facts, put them in the prompt, then generate. It's the backbone of almost every real LLM app. Here's the full pipeline, live.
📚 See it answer from a private knowledge base: https://dev48v.infy.uk/ai/days/day15-rag.html
Retrieve → Augment → Generate
- Embed the question into a vector (last two posts: embeddings + vector DBs).
- Retrieve the top-k most similar chunks from your knowledge base.
- Augment the prompt: system instructions + the retrieved chunks as context + the question.
- Generate — the model answers grounded in those chunks, and can cite them.
In the demo, flip RAG off and the bare model hallucinates or refuses — it never saw your data. Flip it on and the answer is correct and sourced.
Two pipelines
- Offline (indexing): chunk your docs → embed → store in a vector DB.
- Online (querying): embed question → search → build prompt → generate.
RAG vs fine-tuning
RAG adds knowledge (fresh, private, citable) without retraining. Fine-tuning changes behavior/style. Most teams reach for RAG first.
Where it breaks
Bad retrieval → bad answer. That's why reranking, HyDE, and Corrective-RAG (earlier posts) exist — they all make the retrieve step better.
🔨 Build it (chunk → embed → upsert → retrieve top-k → augment → generate + cite) on the page: https://dev48v.infy.uk/ai/days/day15-rag.html
Part of AIFromZero. 🌐 https://dev48v.infy.uk