Last month I spent $200 on OpenAI tokens debugging a RAG pipeline. I had no idea which step was eating my budget. LangSmith would have helped — but I didn't want to send my prompts to a cloud service just to debug locally.
So I built opensmith.
What is opensmith?
A local-first LLM pipeline tracer. Zero cloud.
Zero setup. Everything stays on your machine.
pip install opensmith
How it works
One decorator:
from opensmith import trace
@trace(tags=["rag", "production"])
def pipeline(query: str):
docs = retrieve(query)
return generate(docs, query)
That's it. opensmith captures:
- Function name, inputs, outputs
- Latency in milliseconds
- Token usage per step
- Cost estimate
- Errors with full stack trace
- Parent-child relationships for nested calls
The dashboard
opensmith ui
Opens at localhost:7823. Live WebSocket updates, search, filters, charts. No account needed.
vs LangSmith
| LangSmith | opensmith | |
|---|---|---|
| Setup | Cloud account | pip install |
| Data | Sent to cloud | 100% local |
| Framework | Best with LangChain | Any Python |
| Cost | Free then paid | Free forever |
What's new in v0.1.5
Token budget alerts — never blow your API
budget again:
@trace(token_budget=1000)
def expensive_pipeline():
...
# Prints: ⚠ expensive_pipeline used 1,247 tokens (budget: 1,000)
CLI trace filters:
opensmith traces --q rag --status err --tags production
Auto port detection — if 7823 is taken,
opensmith finds the next free port automatically.
opensmith init — creates a starter config:
opensmith init
# Creates opensmith.json in current directory
Autopatch — zero code changes
from opensmith import autopatch
autopatch(only=["openai", "qdrant"])
Supported: OpenAI, Anthropic, LiteLLM, Qdrant, ChromaDB, Pinecone.
Stats after 5 days
1,500+ PyPI downloads. 16 GitHub stars. Growing organically.
GitHub: https://github.com/shivnathtathe/opensmith
If this saves you time, a ⭐ helps others
discover it. Thank you for 1,500+ downloads 🙏
What would you add to a local-first tracing tool?