If you’ve ever built a RAG system, you’ve likely felt the frustration of the "Mismatch Problem". You ask a perfectly reasonable question, but it returns completely irrelevant documents.
Why? Because your retrieval method is searching based upon your question's language. In the vector world, these two things often don't look alike.
Eg: Users asking questions to retrieve context from a technical documentation (like company's legal policies)
Today, we’re going to master HyDE (Hypothetical Document Embedding)—a technique that flips the script by "hallucinating" the answer before it even touches your database.
📝 What is HyDE?
Instead of taking a user's question and searching for it directly, HyDE follows a three-step dance:
The Hallucination: It asks an LLM to write a "fake" or hypothetical answer to the user's question in document friendly language (using few-shot prompting).
The Embedding: It converts that "fake" answer into a vector.
The Retrieval: It searches your database for real documents that look like that fake answer.
🧩 The Problem It Solves: Asymmetric Retrieval
In standard search, we assume vector(Question) ~ approx vector(Answer).
But in reality, questions are short, curious, and often informal.
Answers are long, factual, and professional.
This is Asymmetric Retrieval.
HyDE turns an Asymmetric problem into a Symmetric one by making the query "look" like the data it’s trying to find.
📍 A Real-World Example: The Legal "Needle"
Imagine you are building a RAG for a law firm. A junior associate asks:
"What happens if a rival company takes over the vendor?"
The Problem: The 5,000-page contract in your database doesn't use the word "rival" or "takes over". It uses professional jargon like "Change of Control Event".
A standard search might fail because these two vectors aren't close enough.
💡 The HyDE Solution:
The LLM generates a "fake" clause:
"In the event of a Change of Control to a Restricted Entity, the Successor shall..."
The system searches for that text.
It immediately finds the correct legal page because the "fake" answer and the "real" document speak the same language.
⚙️ Practical Implementation (The "Professional" Way)
import os
from langchain.chat_models import init_chat_model
from langchain_huggingface import HuggingFaceEndpointEmbeddings
from langchain_community.vectorstores import Chroma
# from langchain_classic.chains import HypotheticalDocumentEmbedder ## Not widely used, since custom functions give more flexibility
from langchain_core.prompts import FewShotPromptTemplate, PromptTemplate
from langchain_core.documents import Document
from dotenv import load_dotenv
load_dotenv()
# 1. Prompt prep: STYLE EXAMPLES (The "Linguistic DNA")
examples = [
{"question": "rival acquisition", "answer": "In the event of a Change of Control to a Restricted Entity..."},
{"question": "sharing info with others", "answer": "Confidential Information shall not be disclosed to any third-party without prior written indemnification..."}
]
example_prompt = PromptTemplate(
input_variables=["question", "answer"],
template="User: {question}\nLegal Style: {answer}"
)
hyde_prompt = FewShotPromptTemplate( ## Class provided by langchain for few-shot prompting
examples=examples,
example_prompt=example_prompt,
prefix="You are a Legal Architect. Translate the query into formal contractual prose.",
suffix="User: {question}\nLegal Style:",
input_variables=["question"],
)
# 2. MODELS & HYDE EMBEDDER
llm_groq = init_chat_model(model="openai/gpt-oss-120b", model_provider='groq', temperature=0)
base_embeddings = HuggingFaceEndpointEmbeddings(
model="sentence-transformers/all-MiniLM-L6-v2", ## this model returns 384 sized vector
task="feature-extraction")
# 3. The DUMMY Documents
# We add diverse sections to ensure the retriever can distinguish between them.
legal_docs = [
# THE TARGET: Change of Control
Document(page_content="Section 45.1: Control Events. A 'Change of Control' occurs if a Restricted Entity acquires 51% of voting shares.",
metadata={"source": "Corp_Bylaws.pdf", "section": "Governance"}),
# THE NOISE: Liability & Indemnity
Document(page_content="Section 10.2: Limitation of Liability. Neither party shall be liable for indirect, incidental, or consequential damages.",
metadata={"source": "MSA_Main.pdf", "section": "Liability"}),
# THE NOISE: IP Rights
Document(page_content="Section 5.4: Intellectual Property. All Work Product created during the Term shall be deemed 'Work Made for Hire' and owned by the Client.",
metadata={"source": "MSA_Main.pdf", "section": "IP"}),
# THE NOISE: Termination
Document(page_content="Section 12.0: Termination for Convenience. Either party may terminate this agreement upon 90 days prior written notice.",
metadata={"source": "Vendor_Agmt.pdf", "section": "Term"}),
# THE NOISE: Confidentiality
Document(page_content="Section 3.1: Non-Disclosure. The Receiver shall protect Confidential Information using the same degree of care as its own proprietary data.",
metadata={"source": "NDA_Standard.pdf", "section": "Privacy"}),
# THE TARGET: Assignment/Successors
Document(page_content="Section 8.9: Successors and Assigns. This Agreement shall be binding upon and inure to the benefit of the Parties and their respective permitted successors.",
metadata={"source": "Corp_Bylaws.pdf", "section": "General"})]
# 4. INITIALIZE VECTOR STORE
vectorstore = Chroma.from_documents(documents = legal_docs,
embedding = base_embeddings,
collection_name = "production_legal_vault")
# 5. TEST THE RETRIEVAL
# Note: The query is vague and uses 0 keywords from the docs.
def hyde_retrieval(query):
# 1. Generate the Hypothetical Document (The "Fake" Answer)
formatted_prompt = hyde_prompt.format(question = query)
hypothetical_doc = llm_groq.invoke(formatted_prompt).content
print(f"--- HYPOTHETICAL DOC GENERATED ---\n{hypothetical_doc}\n")
# 2. Embed the "Fake" Doc and search the "Real" DB
# We use base_embeddings here so the 'math' matches the stored data
results = vectorstore.similarity_search(hypothetical_doc, k=1)
return results
user_query = "What about sensitive or important data's protection?"
final_docs = hyde_retrieval(user_query)
print(f"--- FINAL REAL DOCUMENT FOUND ---")
print(f"Source: {final_docs[0].metadata['source']}")
print(f"Actual Text: {final_docs[0].page_content}")
⌚ When to Use HyDE (and When to Skip It)
✅ Use it when:
Queries are vague or short: If users type "refund" and your docs say "reimbursement protocols," HyDE will bridge that gap.
Terminology Mismatch: Your users are "laymen" and your docs are "experts" (Medical, Legal, Engineering).
High-Stakes Accuracy: When finding the right page is more important than saving a few pennies on API costs.
❌ Skip it when:
Factual/Number Lookups: If a user asks "What was the revenue in 2023?", the LLM might hallucinate a fake number in the hypothetical doc, leading the search to the wrong year.
Latency is Critical: HyDE requires an extra LLM call, which adds 1–2 seconds of "thinking time."
Tight Budgets: Every search now costs extra LLM tokens.
🎯 Summary: Pros & Cons
👍 Pros:
Superior Context: Maps informal intent to formal data.
Zero Keyword Dependence: You don't need exact word matches.
Scalable: Works across thousands of pages without manual tagging.
👎 Cons:
Latency: Adds an extra step to the search process.
Hallucination Risk: A "too-fake" answer can derail the search.
Cost: Increased token usage per query.
Happy coding! Have you tried HyDE in your projects? Let’s discuss in the comments below! 👇