Why I'm Building a Local-First AI Coding Workspace (And How Behavioral Routing Makes It Work)

There's a pattern forming in the AI coding tool space that I think is worth paying attention to.

GitHub paused Copilot Pro+ signups because agentic workloads broke their cost model. Cursor Pro+ is $60/mo and climbing. Claude Code might leave the Pro tier entirely. The common thread: these tools are all cloud-only, which means every user interaction is a cost event on someone else's infrastructure. At scale, the math stops working. Prices go up, access gets gated, and developers end up paying more for less.

I left my role at Wise earlier this year to build a different kind of AI coding tool. One where local inference is the default and cloud is a resource you use intentionally. That tool is called Rada.

This post is a technical walkthrough of how it works under the hood.

The core thesis

Most of what developers ask an AI coding assistant to do doesn't need a frontier cloud model. Refactors, explanations, boilerplate, quick fixes. That's maybe 80% of interactions, and all of it can run on a local LLM.

The remaining 20% (complex architecture decisions, large-scale code generation, multi-file refactors) benefits from cloud models. So the architecture needs to handle both, and it needs to make the transition between them seamless.

The problem with hot-swapping

The naive approach to local AI tooling is to keep multiple models and swap them based on the task. Need a coding model? Load it. Need a general model? Unload the coder, load the general one.

This is a terrible experience in practice. GGUF models at Q4_K_M quantization run anywhere from 4-11 GB in RAM. Loading and unloading them takes time, spikes memory usage, and creates gaps in responsiveness. If you're in a flow state and the tool needs 30 seconds to swap models, you've already lost the thread.

Rada takes a fundamentally different approach.

Behavioral Routing

Instead of swapping models, Rada keeps a single model resident in RAM and adjusts how it behaves based on what the developer is doing. We call this Behavioral Routing.

The system supports three intent modes:

• Refactor: tighten existing code, improve naming, reduce complexity
• Build: generate new code, scaffold features, implement from scratch
• Learn: explain unfamiliar code, walk through logic, teach concepts

When the developer selects an intent (or the system infers it from context), Behavioral Routing adjusts three parameters on the resident model:

System prompt: each intent gets a tailored system prompt that shapes the model's approach. A Refactor prompt emphasizes preserving behavior while improving structure. A Build prompt focuses on completeness and best practices. A Learn prompt prioritizes clarity and step-by-step explanation.

Temperature: Refactor tasks use lower temperature (more deterministic, safer transformations). Build tasks use slightly higher temperature (more creative solutions). Learn tasks sit in the middle (clear but not robotic).

Context window: the system dynamically adjusts how much context gets sent to the model based on intent. Refactoring a single function needs a narrow window. Building a new feature might need broader project context. Learning about a module needs enough surrounding code to give a complete picture.

The result: one model, three distinct behaviors. No load/unload cycle. No RAM spikes. The model stays warm and responsive.

The local model roster

Rada uses a tiered model roster, all GGUF Q4_K_M quantizations:

Which model gets loaded depends on your hardware, not your preference.

Sentinel: RAM-aware model selection

Sentinel is a Rust background process that monitors system memory and determines which model tier your machine can support. The selection is deterministic: Sentinel reads available RAM, checks it against the model ladder, and picks the highest tier that fits without putting the system under pressure.

On a 16 GB machine, you'll get the 7B or 8B models. On 32 GB+, you can run the DeepSeek MoE. The ladder scales up from there as hardware allows.

This is a deliberate design choice. Asking users to pick their own model leads to people overloading their systems or underutilizing their hardware. Sentinel removes that friction. You open Rada and it works with whatever machine you have.

The Autorouter: when local isn't enough

Some tasks genuinely need cloud models. Complex multi-file refactors, architectural decisions that need broad reasoning, large-scale code generation. For these, Rada has the Autorouter.

The Autorouter evaluates the incoming request and routes it to the appropriate cloud endpoint based on task complexity and the intent mode. The cloud model roster is tiered:

Heavyweight: Claude Sonnet, DeepSeek V3 (for complex reasoning tasks)
Workhorse: Gemini Flash, Mistral Nemo (for solid general-purpose tasks)
Micro: Qwen 2.5 Coder 32B, Gemini Flash-8B (for lighter cloud tasks that still exceed local capability)

Cloud routing is managed through OpenRouter, which is already live in production.

Here's a detail that matters: requests routed through the Autorouter consume at 0.5x the normal daily cloud quota rate. If a user manually selects a cloud model, it burns at 1x. This creates an incentive to trust the Autorouter's judgment rather than always reaching for the biggest model. The system rewards efficient routing.

The daily cloud burst quota

Every Rada user gets a daily cloud burst quota. Free tier users can bring their own API keys. Pro users ($19/mo) get 20 daily bursts. Ultra users ($55/mo) get 75.

The quota resets daily (UTC). This is intentional. Monthly quotas create anxiety and hoarding behavior. Daily quotas encourage consistent usage without the fear of burning through your allocation in the first week.

Combined with the 0.5x Autorouter rate, Pro users effectively get 40 cloud interactions per day when they let the system route. That's enough for a full day of coding where cloud is used for the tasks that actually need it.

Why Rust and Tauri

The backend is built in Rust with Tauri as the desktop framework. A few reasons:

Memory matters: when your app is managing local LLM inference, every megabyte of overhead counts. Rust gives predictable, low memory usage. An Electron wrapper eating 500 MB of RAM on top of your model isn't acceptable.

Sentinel needs to be fast: the RAM monitoring process runs continuously. It needs to be lightweight and responsive. Rust's performance characteristics make this straightforward.

Tauri's footprint: compared to Electron, Tauri produces significantly smaller binaries and uses the system's native webview. On a tool where local resource management is the core feature, the framework can't be the one wasting resources.

The frontend is React, which gives us the UI flexibility we need without fighting the framework.

Why this matters now

The AI coding tool market is at an inflection point. Cloud-only architectures worked when usage was lower and providers were subsidizing costs to gain market share. That era is ending.

GitHub pausing Copilot Pro+ isn't a temporary measure. It's a signal that cloud-only AI tooling at consumer price points doesn't have sustainable unit economics. The cost per agentic session is too high.

Local-first isn't about being anti-cloud. It's about using cloud intentionally and keeping the majority of interactions local where they belong. The developer gets a faster, more responsive tool. The provider gets margins that actually work. The dependency on any single cloud provider goes away.

What's next

Rada is heading into closed beta. We're looking for developers who want to test the local-first approach and help calibrate the system with real-world hardware data. Early testers get first right to a lifetime deal on cloud routing.

If this architecture interests you, the waitlist is at userada.dev.

I'll be writing more about the technical decisions behind Rada here on dev.to. Next up: how Sentinel's RAM heuristics work in practice across different hardware configurations.

---

Eli Hadam Zucker is the developer behind Rada. Previously at Wise. Building local-first AI tooling in Rust.