Photo by Steve A Johnson on Unsplash RAG Made Simple: How AI Finds the Right Answers As AI becomes common in enterprises, responses need to be accurate and grounded in business data. Traditional LLMs can be generic or inaccurate because they rely on pretrained knowledge. RAG solves this by retrieving relevant information from trusted sources first, then using it to generate a better response. This code builds a simple RAG pipeline. It loads text from knowledge.txt, splits it into chunks, creates embeddings, stores them in FAISS for search, and uses FLAN-T5 to answer questions using the retrieved context. Step-by-Step Breakdown: Imports and Data Loading : Imports necessary libraries (FAISS for vector search, NumPy, text splitter, embeddings, and transformers). Reads the knowledge base text from knowledge.txt. Text Chunking : Uses RecursiveCharacterTextSplitter to divide the text into 300-character chunks with 50-character overlap, then prints the chunks for inspection. Embedding Generation : Loads the ‘all-MiniLM-L6-v2’ SentenceTransformer model (fast, local, 384-dimensional embeddings) and encodes all chunks into vectors. Vector Store Setup : Creates a FAISS IndexFlatL2 for exact L2 distance similarity search, adds the chunk embeddings (converted to float32), and confirms the index size. RAG Pipeline Function (answer_question): Retrieve: Embeds the user query, searches FAISS for the top 3 most similar chunks. Augment: Combines retrieved chunks into a context and creates a prompt instructing the model to answer using only that context. Generate: Tokenizes the prompt, generates a response using FLAN-T5 (seq2seq model) with constraints (300 max new tokens, temperature 0.7), and decodes the output. 6. Execution : In the main block, prompts for a user question and calls the function to generate an answer. The pipeline ensures answers are grounded in the provided knowledge, with fallback for unknown information. It runs entirely locally (no API calls). For testing, run the script and input a query. import faiss import numpy as np from langchain_text_splitters import RecursiveCharacterTextSplitter from sentence_transformers import SentenceTransformer with open("data/ITSupportKnowledge.txt", "r", encoding="utf-8") as f: knowledge = f.read() text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=50, length_function=len) chunks = text_splitter.split_text(knowledge) print(f"Total chunks: {len(chunks)}") for i, chunk in enumerate(chunks): print(f"--- Chunk {i+1} ---\n{chunk}\n") # 1. Load the embedding model # 'all-MiniLM-L6-v2' is a fantastic, fast, and small model. # It runs 100% on your local machine. model = SentenceTransformer('all-MiniLM-L6-v2') # 2. Embed all our chunks # This will take a moment as it "reads" and "understands" each chunk. chunk_embeddings = model.encode(chunks) print(f"Shape of our embeddings: {chunk_embeddings.shape}") #Step 4: Vector Store with FAISS #We have our vectors. Now we need a database to store them in a way we can search by similarity. It is where FAISS comes in. Don’t be intimidated; it’s just a few lines of code: # Convert to float32 and normalize for cosine similarity search chunk_embeddings = np.array(chunk_embeddings).astype("float32") faiss.normalize_L2(chunk_embeddings) # Get the dimension of our vectors (e.g., 384) d = chunk_embeddings.shape[1] # 1. Create a FAISS index # IndexFlatIP is used with normalized vectors for cosine similarity search. index = faiss.IndexFlatIP(d) # 2. Add our chunk embeddings to the index index.add(chunk_embeddings) print(f"FAISS index created with {index.ntotal} vectors.") # Step 5: Retrieve, Augment, Generate from transformers import AutoTokenizer, AutoModelForSeq2SeqLM # 1. Load a seq2seq model instead of pipeline('text-generation') tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base") generator = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-base") # --- This is our RAG pipeline function --- def answer_question(query): # 1. RETRIEVE # Embed the user's query query_embedding = model.encode([query]).astype("float32") faiss.normalize_L2(query_embedding) # Search the FAISS index for the top k (e.g., k=3) most similar chunks k = 3 distances, indices = index.search(query_embedding, k) # Get the actual text chunks from our original 'chunks' list retrieved_chunks = [chunks[i] for i in indices[0]] context = "\n\n".join(retrieved_chunks) # 2. AUGMENT # This is the "magic prompt." We combine the retrieved context # with the user's query. prompt_template = f""" Answer the following question using only the provided context. Give a short, clear answer in 2 to 3 sentences. Do not repeat information. If the answer is not in the context, say exactly: I don't have that information. Context: {context} Question: {query} Answer: """ # 3. GENERATE # Feed the augmented prompt to our generative model inputs = tokenizer(prompt_template, return_tensors="pt", truncation=True, max_length=768) outputs = generator.generate( **inputs, max_new_tokens=300, temperature=0.7, repetition_penalty=1.2, no_repeat_ngram_size=3 ) answer = tokenizer.decode(outputs[0], skip_special_tokens=True).strip() answer = answer.split("\n")[0].strip() print("Retrieved indices:", indices[0]) print("Scores:", distances[0]) print(f"--- CONTEXT ---\n{context}\n") return answer if __name__ == "__main__": query = input("Ask a question: ") response = answer_question(query) print("Answer:", response) RAG Pipeline Summary: Loads IT support knowledge → splits into 300-char chunks → embeds locally with SentenceTransformer → stores in FAISS index using cosine similarity. When asked a question, it retrieves the 3 most relevant chunks, augments them into a prompt, and generates a 2–3 sentence answer using FLAN-T5 (local model). Everything runs offline — no API calls. RAG Made Simple: How AI Finds the Right Answers was originally published in GoPenAI on Medium, where people are continuing the conversation by highlighting and responding to this story.