Build multi-tenant AI chatbots with persistent memory, streaming, tool calling, and 9 vector DB providers — in 10 lines of Python.
The RAG Boilerplate Problem Nobody Talks About
Every developer building a production RAG chatbot eventually faces the same wall.
You start with a LangChain tutorial. You connect an LLM. You load a PDF. You get a response. It works — and then reality hits.
You need multiple bots for multiple customers. You need their conversation history to survive a server restart. You need real-time streaming responses. You need your bot to call external APIs when documents don’t have the answer. You need to store vectors somewhere other than RAM. You need encryption. You need a REST API so the frontend team can actually use this thing.
What started as a weekend prototype turns into hundreds of lines of infrastructure glue — and none of it is the actual product you are building.
This is the problem LongTrainer was designed to solve.
What Is LongTrainer?
LongTrainer is a production-ready, open-source Python RAG (Retrieval-Augmented Generation) framework published under the MIT License. It is an opinionated, batteries-included abstraction layer on top of LangChain and LangGraph that handles the full production chatbot lifecycle:
- Document ingestion from 15+ sources
- Vector embedding and retrieval across 9 vector database providers
- Multi-tenant bot isolation with per-bot LLM, embeddings, and config
- Persistent conversation memory backed by MongoDB
- Streaming responses — sync and async
- Tool calling and agent reasoning via LangGraph
- Vision and multimodal chat
- Chat encryption at rest
- A built-in FastAPI REST server with zero configuration
pip install longtrainer
# With optional agent/tool-calling support
pip install longtrainer[agent]
Full documentation is available at endevsols.github.io/Long-Trainer.
Why LongTrainer Over Raw LangChain or LlamaIndex?
Here is an honest comparison of what a production RAG system requires you to build yourself versus what LongTrainer provides:
Concern Roll Your Own LongTrainer Multi-bot management Manage state dictionaries per tenant initialize_bot_id() → fully isolated bot Persistent memory Wire MongoDB or Redis manually Built-in MongoDB-backed history Document ingestion Assemble loaders + splitters add_document_from_path(path, bot_id) Streaming Implement astream callbacks get_response(stream=True) yields chunks Tool calling / Agent Build LangGraph graph from scratch add_tool(my_tool) + agent_mode=True Web search augmentation Find, integrate, and maintain web_search=True flag Vision/multimodal Complex multi-modal pipeline get_vision_response() Self-improvement Not a standard concept train_chats() feeds Q&A back into KB Encryption at rest Implement Fernet yourself encrypt_chats=True REST API Build FastAPI server yourself longtrainer serve
The framework operates in two modes:
RAG Mode (LCEL Chain): Fast, deterministic document Q&A using LangChain Expression Language. Best for knowledge base chatbots and document assistants where the document is the authoritative source.
Agent Mode (LangGraph): A full agentic reasoning loop. The bot decides when to query documents, when to invoke tools, and how to chain multi-step reasoning. Best for workflows that require acting on external data.
Quickstart: From Zero to a Working RAG Bot
System Dependencies
Linux (Ubuntu/Debian)
sudo apt install libmagic-dev poppler-utils tesseract-ocr qpdf libreoffice pandoc
macOS
brew install libmagic poppler tesseract qpdf libreoffice pandoc
Initialize
from longtrainer.trainer import LongTrainer
import os
os.environ["OPENAI_API_KEY"] = "sk-..."
trainer = LongTrainer(
mongo_endpoint="mongodb://localhost:27017/",
max_token_limit=32000,
chunk_size=2048,
chunk_overlap=200,
num_k=3,
encrypt_chats=False,
)
Load Documents
LongTrainer supports an extensive range of ingestion sources:
bot_id = trainer.initialize_bot_id()
# Local files — PDF, DOCX, CSV, HTML, Markdown, TXT
trainer.add_document_from_path("contracts/agreement.pdf", bot_id)
# Web URLs and YouTube transcripts
trainer.add_document_from_link(["https://docs.yourapp.com/api"], bot_id)
# Amazon S3
trainer.add_document_from_aws_s3("my-bucket", "folder/data.pdf", bot_id)
# Google Drive
trainer.add_document_from_google_drive(folder_id="1abc...", bot_id=bot_id)
# Confluence wiki
trainer.add_document_from_confluence(
url="https://yourco.atlassian.net",
username="you@yourco.com",
api_key="...",
space_key="ENG",
bot_id=bot_id,
)
# GitHub repository
trainer.add_document_from_github(repo="https://github.com/you/repo", bot_id=bot_id)
# Dynamic injection — any LangChain document loader
trainer.add_document_from_dynamic_loader(MyCustomLoader, {"param": "value"}, bot_id)
Create the Bot and Chat
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
trainer.create_bot(
bot_id,
llm=ChatOpenAI(model="gpt-4o-mini", temperature=0.2),
embedding_model=OpenAIEmbeddings(model="text-embedding-3-small"),
num_k=5,
prompt_template="You are a helpful assistant. Answer only from the provided context. {context}",
)
chat_id = trainer.new_chat(bot_id)
answer, sources = trainer.get_response(
"What are the termination clauses in section 4?",
bot_id,
chat_id,
)
print(answer)
Streaming
# Synchronous streaming
for chunk in trainer.get_response("Summarize the key points", bot_id, chat_id, stream=True):
print(chunk, end="", flush=True)
# Async streaming — for FastAPI and other async frameworks
async for chunk in trainer.aget_response("Explain section 7", bot_id, chat_id):
print(chunk, end="", flush=True)
Multi-Tenancy: Built for SaaS
Every bot created via initialize_bot_id() receives a unique identifier. All associated data — documents, vector embeddings, conversation history, tool registrations, and per-bot configuration — is fully isolated to that ID.
You can run hundreds of bots on a single LongTrainer instance with no cross-contamination:
# Customer A — Legal documents, GPT-4o-mini
bot_a = trainer.initialize_bot_id()
trainer.add_document_from_path("customer_a_contracts.pdf", bot_a)
trainer.create_bot(bot_a, llm=ChatOpenAI(model="gpt-4o-mini"))
# Customer B — Technical docs, Claude, custom embedding
bot_b = trainer.initialize_bot_id()
trainer.add_document_from_path("customer_b_api_docs.pdf", bot_b)
trainer.create_bot(
bot_b,
llm=ChatAnthropic(model="claude-3-5-sonnet-20241022"),
embedding_model=OpenAIEmbeddings(model="text-embedding-3-large"),
)
Bots persist across server restarts. Restore any previous bot with:
trainer.load_bot(bot_id)
Agent Mode and Tool Calling
When retrieval alone is not enough — when your bot needs to act, not just answer — agent mode enables a full LangGraph reasoning loop.
Dynamic Tool Loading (Zero Code)
trainer.create_bot(
bot_id,
agent_mode=True,
tools=[
"tavily_search_results_json",
"wikipedia",
"arxiv",
"PythonREPLTool",
"yahoo_finance_news",
],
)
LongTrainer dynamically imports and initializes any string-based tool
from langchain.agents.load_tools — no manual wiring required.
Custom Tool Registration
from langchain_core.tools import tool
from longtrainer.tools import web_search
@tool
def get_exchange_rate(currency_pair: str) -> str:
"""Fetch the real-time exchange rate for a currency pair like USD/EUR."""
return fetch_rate_from_api(currency_pair)
trainer.add_tool(web_search, bot_id)
trainer.add_tool(get_exchange_rate, bot_id)
trainer.create_bot(bot_id, agent_mode=True)
chat_id = trainer.new_chat(bot_id)
response, _ = trainer.get_response(
"What is the current EUR/USD rate and what does the latest Fed statement say about it?",
bot_id,
chat_id,
)
The agent autonomously decides when to query documents, when to call web search, and when to invoke your custom tool — all within a single turn.
Vector Database Support
LongTrainer treats vector store portability as a first-class concern, supporting nine providers out of the box:
Provider Type Best For FAISS Local / In-memory Development, small scale Pinecone Cloud-native Serverless, large scale Chroma Open-source Self-hosted, fast prototyping Qdrant Open-source High-performance filtering PGVector PostgreSQL extension Existing Postgres infrastructure MongoDB Atlas Cloud Unified database + vector search Milvus Open-source Billion-vector scale Weaviate Open-source Multi-modal, GraphQL Elasticsearch Enterprise Existing ES infrastructure
Each bot can use a different vector store — a meaningful advantage in multi-tenant architectures where different customers may have different infrastructure requirements.
LLM Provider Support
LongTrainer’s Dynamic Model Factory accepts any BaseChatModel implementation:
# OpenAI
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4.1")
# Anthropic Claude
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-3-5-sonnet-20241022")
# Google Gemini
from langchain_google_vertexai import ChatVertexAI
llm = ChatVertexAI(model="gemini-2.5-pro")
# AWS Bedrock
from langchain_aws import ChatBedrock
llm = ChatBedrock(model_id="anthropic.claude-3-5-sonnet-20241022-v2:0")
# Groq
from langchain_groq import ChatGroq
llm = ChatGroq(model="llama-3.1-70b-versatile")
# Ollama (local / air-gapped inference)
from langchain_ollama import ChatOllama
llm = ChatOllama(model="llama3.2")
trainer.create_bot(bot_id, llm=llm)
Per-bot LLM configuration makes LongTrainer well-suited for architectures where difforent customers or use cases warrant different models — GPT-4o for enterprise users, Ollama for on-premise deployments with strict data residency requirements.
Vision and Multimodal Chat
vision_chat_id = trainer.new_vision_chat(bot_id)
response, sources = trainer.get_vision_response(
"What defects are visible in this manufacturing photo?",
image_paths=["inspection_001.jpg", "inspection_002.jpg"],
bot_id=bot_id,
vision_chat_id=vision_chat_id,
)
print(response)
Self-Improving Memory: train_chats()
After a bot accumulates conversation history, you can feed that history back into its knowledge base:
trainer.train_chats(bot_id)
The framework extracts high-quality Q&A pairs from past sessions and re-ingests them as documents. Over time, the bot gets better at answering the specific questions your users are actually asking — a continuous improvement loop that raw LangChain pipelines do not provide out of the box.
Zero-Code CLI and FastAPI Server
LongTrainer 1.2.1 ships with a production-ready CLI and REST API server.
Terminal Chat
# Initialize project
longtrainer init
# Create a bot
longtrainer bot create --prompt "You are a helpful customer support agent."
# Add a document
longtrainer add-doc <bot_id> /path/to/faq.pdf
# Start an interactive chat session
longtrainer chat <bot_id>
REST API Server
longtrainer serve
Starts a FastAPI server at http://localhost:8000 with 18 REST endpoints covering full CRUD for bots, document ingestion, chat session management, and streaming. The Swagger UI is auto-generated at http://localhost:8000/docs.
Endpoint Method Description /health GET Health check /bots POST Create bot /bots/{id}/documents/path POST Ingest file /bots/{id}/chats POST Create chat session /bots/{id}/chats/{chat_id} POST Chat with streaming
The server is Docker-ready and suitable for production deployment behind a reverse proxy.
Complete API Reference
Constructor
trainer = LongTrainer(
mongo_endpoint="mongodb://localhost:27017/",
llm=None, # Default: ChatOpenAI(model="gpt-4o-2024-08-06")
embedding_model=None, # Default: OpenAIEmbeddings()
prompt_template=None,
max_token_limit=32000,
num_k=3,
chunk_size=2048,
chunk_overlap=200,
ensemble=False, # Multi-query ensemble retrieval
encrypt_chats=False, # Fernet encryption at rest
encryption_key=None, # Auto-generated if None
)
Production Tuning
Multi-Query Ensemble Retrieval Enable with ensemble=True. Generates multiple reformulations of each user query and merges the retrieval results — significantly improves recall for ambiguous or conversational queries at the cost of additional LLM calls per turn.
Chunk Strategy The default chunk_size=2048 with chunk_overlap=200 works well for general prose documents. For structured content — tables, code, legal clauses — reduce chunk_size and increase chunk_overlap to avoid splitting semantic units across boundaries.
num_k Tuning Start with num_k=3 for focused Q&A. Increase to num_k=7–10 for synthesis tasks where broader context improves answer quality.
MongoDB Indexing For deployments with hundreds of bots and thousands of conversations, index your MongoDB collections on bot_id and chat_id fields to maintain consistent query performance at scale.
Token Budget max_token_limit=32000 controls the conversation context window. For models with 128K+ context windows, this value can be increased substantially. Monitor document sizes in the memory collection as conversations grow.
Real-World Use Cases
SaaS Multi-Tenant Document Assistant Each customer gets an isolated bot seeded with their own uploaded documents. Conversation history persists across sessions. LongTrainer’s bot_id / chat_id isolation model makes this architecture a few lines of code rather than an engineering project.
Enterprise Internal Knowledge Base Load Confluence wikis, GitHub repos, internal PDFs, and S3 buckets into a single bot. Enable ensemble=True for better recall on ambiguous queries. Enable encrypt_chats=True for compliance requirements.
AI Customer Support Agent Use agent mode with web search and a CRM lookup tool. The bot retrieves from product documentation, checks live ticket status via tool calls, and returns grounded answers.
Research Assistant with Continuous Improvement Feed academic PDFs into a bot. Run train_chats() periodically to re-ingest high-quality Q&A pairs from past sessions. The bot improves incrementally without retraining.
On-Premise Deployment for Data Residency Use ChatOllama as the LLM with a local FAISS store. No data leaves the premises. longtrainer serve provides the REST interface for internal applications.
Getting Started
PyPI: pip install longtrainer
GitHub: github.com/ENDEVSOLS/Long-Trainer
Documentation: endevsols.github.io/Long-Trainer
Open Source Tools: endevsols.com/open-source/longtrainer
If LongTrainer saves you meaningful engineering time, consider starring the repository and sharing it with your team.
Tags: #Python #MachineLearning #LangChain #RAG #AI #ChatBot #OpenSource #LLM #NLP #GenerativeAI #LangGraph #VectorDatabase #ArtificialIntelligence #MLOps