This part of the series is a “mind-blower” for most developers. Here is the Medium-style article for Part 5, focusing on Dynamic Model Routing and Runtime Awareness.

In our previous articles, we’ve always defined our agent’s “brain” once: model = init_chat_model("gpt-4o")

While this works, it’s inefficient. If a user asks a simple question like “How do I list pods?”, using a powerful, expensive model like GPT-4o is overkill. But if the production database goes down, a smaller, faster model (like Phi-4 or Llama-3–8B) might not be smart enough to handle the crisis.

Today, we’re going to build a “Panic Router” — an agent that dynamically swaps its own brain based on the urgency of the problem.

Static vs. Dynamic Models

• Static models: Configured once at startup. They remain the same whether the user is asking for a joke or reporting a security breach.
• Dynamic models: Selected at runtime based on context. This is the holy grail of cost optimization and performance.

By using LangChain’s wrap_model_call middleware, we can intercept the agent's request to its LLM and "hot-swap" the model before the call even happens.

1. Defining the Junior and Senior “Personas”

First, let’s initialize two models. A small, local model for routine tasks, and a more capable (and likely expensive) model for emergencies.

junior_model = init_chat_model("ollama:phi4-mini", temperature=0) # Fast & Cheap
senior_model = init_chat_model("ollama:qwen3:4b", temperature=0) # Smart & Deep

2. The “Panic Router” Middleware

This is the core logic. We use the @wrap_model_call decorator to scan the incoming user message for critical keywords like "crash," "fire," or "down." If it looks like an emergency, we escalate.

from langchain.agents.middleware import wrap_model_call, ModelRequest
@wrap_model_call
def panic_router(request: ModelRequest, handler):
    last_message = request.state["messages"][-1]
    critical_keywords = ["crash", "down", "emergency", "500", "critical", "fire"]
        user_text = last_message.content.lower()
    is_emergency = any(word in user_text for word in critical_keywords)
    if is_emergency:
        print("🚨 Escalating to SENIOR SRE.")
        request = request.override(model=senior_model)
    else:
        print("✅ Handling with JUNIOR SRE.")
        request = request.override(model=junior_model)
    return handler(request)

By using request.override(model=...), we are essentially performing a brain transplant in the middle of a conversation.

3. Runtime Awareness: Context and State

One reason this is so powerful is that LangChain 1.2 runs on the LangGraph Runtime. This gives your middleware and tools access to “Runtime Objects” like:

• Context: Information like User ID, DB connections, or environment variables.
• Store: Long-term memory that persists across different chat sessions.

You can use this runtime information inside your tools to check if a system is struggling before the model even asks, allowing for truly proactive AI operations.

4. Seeing it in Action

When we deploy our “DevOps Dispatcher,” it acts like a triage nurse:

Scenario A: Routine Inquiry

• User: “How do I list pods?”
• Router: No critical keywords found. Handled by Junior Model (Phi-4).
• Cost: Minimal. Speed: Instant.

Scenario B: The Server is on Fire

• User: “The production database is DOWN! Help!”
• Router: Keyword “DOWN” detected! Escalating to Senior Model (Qwen-3).
• Result: The smarter model takes over to troubleshoot complex networking and database issues.

Conclusion: Selective Intelligence

The future of AI engineering isn’t just about finding “the best” model; it’s about selective intelligence. By building a dynamic router, you get the best of both worlds: the low cost and speed of small models for 90% of requests, and the raw power of large models when the stakes are high.

In the final part of our series, we’ll see how to combine Dynamic Models with Summarization Middleware to build the ultimate production AI agent.

💬 What do you think?
Drop your thoughts, questions, or suggestions in the comments below!

Check out my YouTube channel for more exciting content! [YouTube Channel Link — Harsha Selvi]

Disclaimer: This text has been rephrased using AI tools, and some parts are derived from various sources to provide a comprehensive overview.

#AI #LangChain #MachineLearning #AIEngineering #Python #CloudNative #DevOps #Architecture