TL;DR — Our goal was a free-form agent—like Cursor or Claude Code—where users start anywhere, ask anything, and never march through a fixed pipeline. Getting there meant progressively moving responsibility off the prompt and onto the harness: first sequencing and state, then lifecycle and triage. The conversational freedom at the top was only safe because we kept adding structural guardrails underneath.

We wanted twio to feel like Cursor, but for mortgage brokers: a free-form agent you simply talk to. Jump into the middle of a refix, ask "when does their rate expire?", paste a customer's email, and let the agent carry on. No fixed pipelines, no "step 1 of 4"—just work the way it actually arrives.

Our first version was the exact opposite of this. It could complete a refix end-to-end, but only if the broker followed the script's exact sequence. Start in the middle, ask a side question, or reply three days later, and the system would break.

This is the story of how we chased that conversational freedom across three architectures. On the surface, each rebuild looked like a structural change. Underneath, it was a steady offloading of responsibilities—stripping away what the LLM was bad at holding, and handing it to the harness.

(To understand the evolution, just know that a **refix—renewing a mortgage rate—sounds like a 4-step linear process, but in reality, it's fragmented, non-linear, and spans days. Keep that in mind.)

Architecture 1: The Monolith Script (and the Illusion of Control)

When we started, we had no prior experience with LLM harnesses. Freedom was the goal, but first we had to answer a simpler question: Can an LLM handle a real mortgage workflow end-to-end at all?

Our first architecture was one giant prompt driving the entire refix. It looked roughly like this:

You are a mortgage assistant. To do a refix:
1. Look up the customer by name; if several match, ask which one...
2. Ask for the new rate and term — unless the customer emailed...
3. Build the proposal. Repayment = P&I, unless interest-only...
4. Fill the lender form. ANZ uses fields X/Y; ASB uses...
5. Draft the sign-off email. Warm but professional...
...and ~300 more lines of rules, edge cases, and tone notes.

It was simple, predictable, and legible. Then it met production reality, and fractured under two fatal flaws:

1. Cognitive Overload: The longer the workflow, the more the prompt ballooned with edge cases, scattering the model's attention. It assumed a fixed sequence, but brokers don't work that way.
2. Tangled Coupling: One prompt carrying fetch, validate, draft, and edge-case logic became an entangled wall of text. You couldn't test "build the proposal" without running the whole machine.

The recurring disruption that exposed these flaws: "The customer replies three days later." In the monolith, there was simply no place for this. The run was over, or it was blocked waiting for a turn that wasn't coming.

The Diagnosis: A monolithic prompt fuses three things that should never mix.

Fused together, changing one risks breaking the others.

The mandate for our first refactor was clear: Pull these three apart, and stop hardcoding the sequence.

Architecture 2: Planner + Steps (Decoupling Flow and State)

We shattered the monolith into Steps. A step is a small, single-purpose agent with a typed contract: a focused prompt, a whitelist of tools, and strictly defined I/O schemas for state.

A Planner produced an ordered sequence, and an orchestrator executed them. This cleanly separated the monolith's responsibilities:

The planner orders typed steps; each runs its own small, scoped LLM prompt.

The Breakthrough: Context as Memory

The most critical insight of this era was how we handled state. Steps didn't get the whole conversation history; they got a scoped view.

refix_proposal reads parties, writes proposal, and never sees the rest. This isn't just access control; it's memory management. A model reasons far more accurately over a small, relevant context than a massive one. Scoping the view didn't just tidy the code—it made the steps sharper.

Architecture 2 was a massive leap forward. We shipped a lot of features on it. But it carried a hidden, fatal assumption: It assumed work is synchronous and continuous.

To start work, the broker had to land on a homepage and select the workflow type from a dropdown (e.g., "Refix"). Choosing "Refix" launched the fixed plan. That demands the broker knows the shape of the work before anything runs. (You don't pick "refactor" from a menu before Cursor listens—you just start typing).

Furthermore, the plan assumed run-to-completion. So, our recurring disruption returned: The customer replies three days later. By then, the pipeline had finished or stalled. We tried bolting on a separate "inbox" to triage incoming items, but it felt like stapling an async patch onto a synchronous system. We deleted it.

The Diagnosis: A refix is not a workflow you execute. It's a long-lived case, fed by disconnected events over days.

The mandate for the next refactor: Stop making the broker declare the work up front. Stop pretending work runs to completion.

Architecture 3: The Open Case (Event-Driven Paradigm)

We collapsed the dropdown menu of workflow types into one universal Case. Every inbound event—email, chat message, late reply—flows into the same entry point. The first thing that runs is the planner, but it now has a completely different job: Triage.

Instead of sequencing a known workflow, the planner looks at a single event and picks a path:

Triage routes each event — answer directly, wake a case, or compose steps. A late reply just re-enters triage.

Three mechanisms make this architecture work:

1. Long-Lived Cases with Implicit Waiting
After acting, a case goes quiet. In our system, the absence of new messages is the waiting state—there is no dangling pipeline. What broke Architecture 1 and stalled Architecture 2 is now trivial.

2. First-Class "Do Nothing" Path
When a broker asks "when does their rate expire?", the triage planner answers it directly and stops. No four-step plan is generated. Often, no plan is the right plan.

3. Declarative Playbooks
Domain knowledge moved out of code and into Markdown playbooks. This allows mortgage experts to edit the "how" without touching the engine code.

Guarding the Freedom: Data-Flow Validation

Handing a language model the freedom to compose its own step sequences is dangerous. We put a hard floor under it: a data-flow validator that rejects an incoherent plan before a single token of work is spent.

If the planner emits refix_proposal before parties, the validator hands back a precise error, and the planner self-corrects.

This introduces a brilliant mechanism we call Soft Reads vs. Hard Reads:

• Hard Read (Correctness): An ordering constraint. form_filler must wait for parties. If it's missing, the validator blocks it.
• Soft Read (Composability): Visibility-only. form_filler can see refix_proposal if it exists, but won't force it into the plan. This lets the step be dropped into a one-off request without dragging the whole refix sequence along.

One reads declaration scopes the step's memory and acts as an edge in the dependency graph. One declaration, two jobs.

This is where twio finally felt like the free-form agent we set out to build. It’s the synthesis of our journey: Freedom at the top, structure at the bottom, knowledge in prose.

The Underlying Pattern

Step back, and the three architectures align on a single axis: How much we asked the LLM to hold in its working memory.

Each refactor took one responsibility off the prompt and gave it to the harness:

• Script → Planner+Steps: Offloaded sequencing and state.
• Planner+Steps → Open Case: Offloaded lifecycle and triage.

That is what building with the grain of the LLM meant for us. Not a clever prompt. A steady accounting of what the model is bad at holding—durable state, long-running control flow, cross-day triage—and handing those to a harness built to hold them.

What's Still Hard

Top-level freedom introduces new failure modes. The planner can match the wrong playbook. It can compose a valid but suboptimal plan. Prose playbooks can drift from what the steps actually execute. Our guardrails catch a majority of this, but this is a live frontier, not a solved problem.

The through-line of this whole journey is that you don't arrive at the architecture—you get pushed into it by reality. Ultimately, the customer who replies three days late wrote more of our architecture than we did.

Architectural Takeaways

If you are building long-lived, agentic systems:

1. Don't make the model hold what the harness holds better. Offload sequencing, durable state, and lifecycle management to traditional code.
2. Treat context as memory. Strictly scope each step's view. Models reason better over small, relevant contexts.
3. Model the arriving unit as an event, not a task. Work is fragmented; your architecture should expect interruption.
4. Build a first-class "just respond" path. Forcing a workflow when a direct answer suffices destroys the user experience.
5. Earn freedom with guardrails. Use static validation (like our data-flow checker) to make self-composing agents safe.

We're building twio, an AI assistant for mortgage brokers. If you're wrestling with the same questions about long-lived, event-driven agent architectures, we'd love to compare notes.