Introduction

As enterprise adoption of LLMs grows, inference costs, hallucinations, and retrieval inefficiencies are becoming major production challenges.

Traditional vector-based Retrieval-Augmented Generation (RAG) improves grounding, but it still struggles with multi-hop reasoning and relationship-aware retrieval.

For the TigerGraph GraphRAG Inference Hackathon, our team built a complete biomedical GraphRAG inference system that compares:

• LLM-only inference
• Basic RAG (Vector + LLM)
• GraphRAG (Knowledge Graph + LLM)

across latency, token usage, cost, grounded accuracy, and reasoning quality.

Our goal was simple:

Can GraphRAG reduce token usage while maintaining grounded and explainable answers?

Main benchmarking dashboard comparing LLM-only, Basic RAG, and GraphRAG pipelines.

🔗 GitHub Repository:
https://github.com/SIDHANTH-S/graphrag-inference-system

🌐 Live Demo:
http://52.172.150.0:3000/

🎥 Demo Video:
https://drive.google.com/file/d/1CKCUYpRbdjh9qdTHKyu5V2V8J5c0lgRr/view?usp=sharing

Why We Built This

LLMs are powerful, but production AI systems face several challenges:

• Hallucinated answers
• Expensive context windows
• Retrieval noise
• Weak explainability
• Difficulty performing multi-hop reasoning

Basic RAG pipelines solve part of the problem by retrieving semantically similar chunks from vector databases.

However, semantic similarity alone is often insufficient for domains like biomedicine, where relationships between drugs, diseases, enzymes, and pathways are highly structured.

This is where GraphRAG becomes powerful.

Instead of retrieving only semantically similar text, GraphRAG retrieves entities and relationships from a structured knowledge graph, enabling explainable and relationship-aware reasoning.

System Architecture

Our platform combines:

• FAISS for semantic vector retrieval
• TigerGraph for structured biomedical relationships
• LLM-based entity extraction and answer synthesis
• A benchmarking dashboard for evaluation and analytics

End-to-End Architecture

The Three Pipelines

1. LLM-Only Pipeline

This serves as the baseline pipeline.

The user query is sent directly to the LLM without any retrieval or grounding.

Advantages:
• Fast
• Simple

Limitations:
• High hallucination risk
• No evidence grounding
• Poor explainability

2. Basic RAG Pipeline
The Basic RAG pipeline retrieves semantically similar chunks using FAISS embeddings.

Pipeline flow:

Query
→ Embedding generation
→ Vector retrieval
→ Context injection
→ LLM answer generation

Advantages:
• Better grounding than pure LLM inference
• Reduced hallucinations

Limitations:
• Retrieval noise
• Weak relationship understanding
• Difficulty with multi-hop reasoning

3. GraphRAG Pipeline

The GraphRAG pipeline combines semantic retrieval with structured graph traversal.

The workflow includes:

• Query entity extraction
• Entity-to-graph resolution
• Multi-hop graph expansion in TigerGraph
• Evidence fusion
• Grounded answer synthesis

This enables the system to retrieve not only semantically similar text, but also biologically meaningful relationships.

Biomedical Dataset and Knowledge Graph Construction
We used PubMed-style biomedical literature from the MedRAG dataset hosted on Hugging Face.
Dataset Source : https://huggingface.co/datasets/MedRAG/pubmed

The ingestion pipeline performs:

• Document chunking
• Biomedical entity extraction
• Relation extraction
• Dense embedding generation
• TigerGraph vertex/edge creation
• FAISS index construction

The system extracts biomedical entities such as:

• Drugs
• Diseases
• Genes
• Side effects
• Anatomical entities

and stores their relationships in TigerGraph for graph-based retrieval.

High-throughput biomedical ingestion pipeline.

Benchmarking and Evaluation

One of the main goals of this project was not just building GraphRAG, but scientifically evaluating it.

Our dashboard compares:

• Token usage
• Latency
• Estimated API cost
• Grounded accuracy
• BERTScore
• LLM-as-a-Judge evaluation

Example Query: Causal Biomedical Reasoning
One benchmark query asked:

“What is the causal path from alloxan to arteriosclerosis?”

Expected reasoning:

Alloxan
→ causes
Diabetes
→ increases
Arteriosclerosis

The LLM-only pipeline generated a plausible but unverified answer.

Basic RAG retrieved semantically relevant evidence but struggled with structured causal reasoning.

GraphRAG successfully combined semantic retrieval with graph-grounded biomedical relationships to generate a grounded causal explanation with supporting evidence.

Key Results

Across evaluation queries, our GraphRAG pipeline achieved:

• ~52% average token reduction
• ~58% retrieval token savings
• ~61% estimated API cost reduction
• Strong grounded biomedical reasoning
• Improved explainability through graph traces

One of the most important findings was that GraphRAG reduced unnecessary retrieval context while maintaining answer quality through structured graph relationships.

GraphRAG Benchmark Highlights

✔ ~52% average token reduction
✔ ~61% estimated API cost savings
✔ Grounded biomedical reasoning
✔ Multi-hop graph-based retrieval
✔ Explainable evidence-backed answers

What We Learned

One of our biggest takeaways was that GraphRAG is not simply “better retrieval.”

Its real strength comes from:
• structured reasoning
• relationship-aware retrieval
• explainability
• context compression

This becomes especially valuable in biomedical AI systems, where trust, traceability, and multi-hop reasoning are critical

Conclusion

As enterprise AI systems continue to scale, inference efficiency and explainability will become increasingly important.

This project demonstrated that GraphRAG can reduce retrieval overhead while maintaining grounded and explainable reasoning through structured biomedical knowledge graphs.

The combination of vector retrieval and graph traversal opens exciting possibilities for production-grade GenAI systems that are not only accurate, but also interpretable and cost-efficient.
This project was developed as part of the TigerGraph GraphRAG Inference Hackathon.