All tests run on an 8-year-old MacBook Air.
I've built tools with both Gemini API and local LLMs (via Ollama). They're solving different problems.
Here's the honest comparison after shipping both.
Gemini API
What it's good at:
- Complex reasoning over long context (stack traces, multi-file logs)
- Up-to-date knowledge of libraries and frameworks
- Zero setup for the user — just an API key
- Thinking models for tricky causality chains
What it's bad at:
- Privacy-sensitive data (logs might contain PII)
- Offline use
- Free tier rate limits under heavy use
- Latency over slow connections
Best for: Developer tools where the data isn't sensitive and reasoning quality matters more than privacy.
Local LLM (Ollama)
What it's good at:
- Complete privacy — nothing leaves the machine
- Works offline
- No API key, no rate limits, no cost per token
- Instant response once the model is loaded
What it's bad at:
- Requires the user to install Ollama separately
- Model quality is lower than Gemini for complex reasoning
- Slow on older hardware (8-year-old MacBook Air struggles with 7B+ models)
- Large download (~4GB for a decent model)
Best for: Tools handling sensitive data where privacy is non-negotiable.
What I actually shipped
HiyokoLogcat → Gemini API
Logcat analysis benefits from Gemini's deep knowledge of Android internals. The thinking model traces causality chains that a local 7B model misses. Logs do contain some PII, but the privacy filter handles the worst of it.
HiyokoLogcat (future) → user's choice
The ideal setup: Gemini by default, with an option to switch to a local model for sensitive projects. Not shipped yet — but the architecture is designed for it.
The practical decision tree
Does the data contain sensitive information?
Yes → Local LLM (or heavy PII filtering before Gemini)
No → Gemini API
Does quality of reasoning matter more than speed?
Yes → Gemini (especially thinking models)
No → Local LLM (faster, cheaper)
Will users accept installing Ollama?
Yes → Local LLM is viable
No → Gemini API only
The honest answer
For most developer tools: Gemini API with a privacy filter. The quality difference is significant, the free tier is generous, and users don't want to install a 4GB model to try your app.
For tools handling medical records, financial data, or enterprise logs: local only, no exceptions.
HiyokoLogcat is free and open source → github.com/hiyoyok/HiyokoLogcat
X → @hiyoyok