TL;DR: A customer asked my AI sales bot "what do you have?" and the bot listed product categories the store doesn't sell. My instinct was to rewrite the search router. I spent twenty minutes about to do exactly that. Then I traced where the hallucinated category list was actually coming from: not the search results, not the database, not the router. It was coming from the store's "About" text — which the system prompt was injecting as Store: ${store.description}. The model read that label as a catalog header and treated the marketing copy as inventory truth. The fix was renaming one variable string from Store: to About the store (brand voice / background — NOT a product catalog): and adding one CRITICAL rule. Zero changes to the architecture.

The Bug

I run a multi-tenant AI sales chatbot platform. One of the test stores sells men's casual clothing — shirts, pants, the basics. Its description field, the marketing blurb the merchant types on signup, reads something like:

"Modern men's wardrobe. From sharp business shirts to weekend essentials, suits, shoes, and everything in between."

Standard SEO-friendly copy. Reads fine on the storefront page.

A test customer asks the chatbot:

"Hey, what do you have?"

The bot replies:

"We've got a full men's wardrobe — business shirts, weekend essentials, suits, shoes, and everything in between. What are you in the mood for?"

Customer:

"Great, I'll take a suit."

The store has zero suits. Has never sold a suit. The product table has thirty-four rows; none of them are suits. The bot just promised something the catalog can't deliver. The customer escalates, asks for sizing, and now there's a trust break two messages into the conversation.

I have seen this kind of bug before. I had a whole architecture in place to prevent exactly this.

The Architecture I Was Sure I'd Have to Rewrite

When the customer's message hits a generic phrase like "what do you have" or "show me everything," my chat router doesn't call a freeform "describe the store" prompt. It branches into a dedicated path that pulls the actual product table, builds a category breakdown — { "Shirts": 18 items, $20-$60 }, { "Pants": 12 items, $30-$80 } — and feeds that into the response model as the source of truth.

The architecture is deliberate. I wrote about it before: prompt engineering controls tone, architecture controls behavior. If you want the model to never invent a product, don't beg it not to; give it search results and a tool contract that says "you can only reference what came back from this call." The grounded-LLM playbook.

So when I saw the bot recite suits and shoes for a store that has neither, my first instinct was the obvious one. The architecture must have broken. Either:

1. The generic-phrase detection isn't firing, so we're falling through to the freeform path where hallucinations are possible.
2. The category breakdown is returning wrong data — maybe pulling from another store, maybe miscategorizing.
3. The search results are being clobbered somewhere between the SQL and the response prompt.

I started reading the router code with the intent to rewrite it. I had a branch open and a commit message half-typed before I stopped and did one thing first: I read the actual system prompt that was being sent to the model.

Where the Suits Came From

This is the relevant slice of the response-call system prompt as it was being assembled:

const desc = store.description ? ` Store: ${store.description}` : "";
const typeText = store.store_type ? ` Type: ${store.store_type}.` : "";
const countryText = store.country ? ` Location: ${store.country}.` : "";

const systemPrompt = `
You are the sales assistant for ${store.name}.${desc}${typeText}${countryText}
Search results for "${query}":
${searchResults}
...
`;

Look at the line that builds desc. The label is the word Store: followed by whatever the merchant typed into their description field.

Now look at what the model sees, in order:

You are the sales assistant for Diwan.
Store: Modern men's wardrobe. From sharp business shirts to weekend essentials, suits, shoes, and everything in between.
Type: Clothing & Fashion.
Location: Palestine.
Search results for "what do you have":
{ category_overview: { "Shirts": 18 items, "Pants": 12 items } }
...

The architectural defense — the real category overview — is there, lower in the prompt. It's correct. It's accurate. But two lines above it, there's another block of text labeled Store: listing categories that look like inventory: "shirts," "suits," "shoes."

The model has to decide which of those two sources to trust. The architecture was correct. The labels weren't.

The word Store: is not specific. The model doesn't know it's marketing copy. It reads exactly like the kind of label that introduces an inventory list, because in training data, structured labels followed by category-shaped text usually are inventory lists. Every Shopify product CSV header. Every catalog JSON. The model is doing exactly what its training pulls it toward.

The marketing blurb wasn't being treated as marketing. It was being treated as a catalog because it had been labeled like one.

The Fix: Two Lines

There was no architectural change. The router stayed. The search results stayed. The category-overview path stayed. Two edits to the prompt construction:

Edit one — relabel the injection:

const desc = store.description
  ? ` About the store (brand voice / background — NOT a product catalog): ${store.description}`
  : "";

The model now reads the description with an explicit epistemic frame. This text exists, but it is brand voice. It is not inventory. There is a different source for inventory below.

Edit two — add a CRITICAL rule to the response prompt:

CRITICAL: When the customer asks what you have / what you sell / your
catalog / "شو عندك" / "إيش عندكم" "What do you have — list ONLY categories that appear
in the search results. NEVER enumerate categories from the store
background or description text. The background is brand voice; the
search results are inventory truth.

That's the entire fix. Same architecture, same database, same router branches, same tool contract. The bug closed. The bot stopped offering suits the store doesn't sell.

Architecture vs Prompt Is the Wrong Dichotomy

There's a clean-sounding mental model that goes: "if the bug is the model behaving badly, change the architecture; if the bug is the model sounding wrong, change the prompt." I've written and quoted versions of that myself.

It's not wrong, exactly. It's just not the right axis when you're sitting in front of an actual bug, three minutes from typing git checkout -b rewrite-search-router.

A better question to ask first:

Where, in the bytes I send the model, does the wrong information live?

Not "is my architecture sound." Not "is my prompt strict enough." Where, literally, on the screen, are the suits coming from?

In my case, the suits were in the prompt — in a string I'd inserted myself, with a label that the model was perfectly entitled to interpret as a catalog. The architecture was clean. The search was clean. The defense was clean. I just hadn't been careful about what frame I gave the model for each block of context I passed in.

The general pattern, which I now check on every grounded-LLM bug:

1. Trace the output back to a span of bytes in the prompt. Not metaphorically — literally find the substring the model echoed. Is it from searchResults? From store.description? From an example in a few-shot block? From an old conversation summary you forgot was being passed?
2. Look at the label that introduces that span. Store: is not a label, it's a noise word. About the store (brand voice / background — NOT a product catalog): is a label. Specificity here is grounding.
3. Check whether another span in the same prompt contains the correct answer. If yes, the bug is precedence, not absence. The model has both truths in front of it and picked the wrong one because the wrong one had higher epistemic weight from its labeling.
4. Only then ask if the architecture needs changing. Usually it doesn't.

The first time I ran this checklist, the "two-line fix" only existed because I'd already written the architectural defense months earlier. The category-overview path was the truth I needed the model to use. The prompt was just calling something else "Store:" right above it and letting the model decide.

The Inversion

I've published before that prompt engineering controls tone and architecture controls behavior. That's still true. But there's a second half I want to write down, because I keep relearning it:

Architecture builds the truth. The prompt decides whether the model believes it.

You can have a flawless retrieval pipeline, a tool contract, a typed search result, a JSON-mode response constraint — and the model will still output a hallucination if the prompt above the truth says, in any voice, "here's the inventory" while pointing at the wrong block.

The two layers aren't in opposition. They're stacked. Architecture is what you make available to the model. The prompt is how you label what you made available. If the labels are vague, the model fills in the meaning from its training, which usually means it picks the most common interpretation — and the most common interpretation of Store: followed by category-shaped prose is "this is the store's inventory."

When the bug looks architectural, check the prompt. When the bug looks like a prompt problem, check what context is reaching the model. The bug almost always lives at the seam between the two, not inside one of them.

The Takeaway

You don't have to choose between "fix the architecture" and "fix the prompt." That dichotomy will burn afternoons.

Ask one question before you reach for either tool: where, in the bytes I'm sending, does the wrong answer come from?

For me, it was the marketing description. Wearing a catalog label. Sitting two lines above the real catalog. The model wasn't wrong to read it that way. I was wrong to label it that way.

The fix was a string rename. Twenty minutes of diagnosis, eight characters of code. The architecture I almost rewrote was already correct.