The default answer to "how do I give my LLM agent memory" has, for about two years now, been some variant of "set up a vector DB and embed conversation chunks." I've shipped that pattern across half a dozen production agents and watched it underperform every single time. This post is about why structured key-value recall plus MCP-style memory servers usually beats vector embeddings for the actual jobs agents need memory for, and the specific failure modes that drove me to stop reaching for a vector DB first.
The thesis is narrower than it sounds. Vector embeddings work well for what they were designed for: semantic retrieval over large unstructured corpora. They are routinely misapplied to agent memory because the tooling is mature and the conference-talk version of agent architecture says "RAG everything." What agents actually need most of the time is something simpler and harder: structured persistent state with reliable, consistent recall.
Three concrete failure modes I've hit across deployments.
The classic vector-DB memory pattern is: embed every conversation message, retrieve the top-k most similar to the current turn, stuff them into the system prompt. This works in toy demos. In production it produces what I think of as "drift hallucinations", the agent confidently cites facts from a vector hit that's only loosely related to the current turn.
Concrete example. A customer-support agent had vector memory of past tickets. User opens a new ticket about "I can't log in." Vector search returns three past tickets containing "login", including one from a year ago about a different feature that was renamed. The agent confidently tells the user to "check the workflow flag, which solved this in your previous ticket." The flag had been deprecated for nine months. The user spent twenty minutes looking for a UI element that no longer existed.
The vector DB did exactly what it was designed for. It retrieved semantically-similar items. The problem is that "this user's last login attempt was five days ago" and "this user's deprecated workflow flag was relevant in February 2024" are both valid retrievals, and the agent has no way to distinguish "current relevant fact" from "historical artifact."
A structured KV store with explicit recency markers and explicit fact lifecycle ("this fact was true on 2024-02, this fact is current") doesn't have this failure mode. You query by key, not by similarity. If the fact isn't current, it isn't there.
Most agent memory fits in a few hundred bytes per user. The user's name, current preference settings, last few interaction summaries, ongoing project context. A vector DB is wildly the wrong tool for this scale of data. You're storing 1,536-dimension floats to index 200 bytes of structured state, paying 30x the storage and 100x the query latency to retrieve information that a single key lookup would return in microseconds.
This sounds obvious and yet I keep finding production agents where the single most-queried memory fact ("what is the user's preferred response format?") is sitting behind a vector search instead of behind a key.
Vector DB defenders will say: "well, you'd just store that as metadata on the embedding." Sure. Now you have a key-value store with extra steps and a vector index you don't need.
Agents need to write to memory often: every time the user states a preference, every time a fact changes, every time a session wraps up. Vector DBs handle writes, but the lifecycle is awkward.
Take a simple case: the user says "actually, my email is [email protected], not [email protected] as I said before." A vector DB will store both as embeddings. Future retrieval will return both. The agent now has two contradictory facts and no first-class way to mark one as current and the other as historical. You can engineer around this (add a "superseded_by" field, filter on it at retrieval, write your own conflict resolution logic), but at that point you've reinvented a structured database with a vector index awkwardly bolted on.
A structured KV store handles this naturally. user:jane:email gets overwritten. There's only one current value. Old values are explicit history, not retrieval noise.
The phrase covers four distinct workloads, and they have different optimal storage:
The mistake is treating all four as one workload and reaching for a vector DB. In practice you want different stores for different memory types, with the agent's tool layer doing the routing.
The Model Context Protocol gave us a clean way to expose memory to agents without coupling it to a specific store. A memory MCP server is a thin tool layer the agent calls (memory.recall, memory.set, memory.list_recent) backed by whatever storage actually fits the data: KV for structured, vector for semantic, append-only log for episodic.
This pattern has two practical benefits over "embed-and-retrieve":
First, the agent has explicit affordances. Instead of hoping the right embedding gets retrieved, the agent decides "I need to look up the user's stored preferences" and calls memory.recall("preferences"). The decision is in the agent's reasoning, not in the embedding-similarity black box.
Second, you can mix storage backends without changing the agent. The MCP layer hides the implementation. We've shipped one project where structured user state was a Postgres table, episodic session summaries were a Redis sorted set, and semantic doc retrieval was a Qdrant index, all behind one MCP memory server. The agent didn't care.
Memnode (full disclosure, my project) is one such server, focused specifically on the structured + episodic side. There are others. The point isn't memnode specifically; the point is that the MCP-server pattern lets you pick the right storage per workload instead of forcing everything through a vector DB.
To be clear: I'm not saying don't use vector DBs. I'm saying don't make them the default for agent memory. The use cases where vector retrieval is genuinely the best primitive:
The pattern is: vector DB when the corpus is large, the queries are fuzzy, and exact key lookup wouldn't work because keys aren't known in advance.
If your agent's memory is "small structured state per user" or "the last 50 messages of a conversation," that's not what vector DBs are for.
A diagnostic question that's served me well: write down the five most common memory-related questions your agent asks itself.
If they look like:
→ KV-style structured memory. Don't use a vector DB.
If they look like:
→ Vector retrieval. Use a vector DB.
If they look like:
→ Hybrid: episodic log + on-demand summarization, possibly with a small vector index over the summaries (not the raw turns). Reach for the MCP-server pattern so you can tune the backend per query type.
For new agent projects, my default stack is:
The cost has been lower (no vector DB to operate), the failure modes have been more predictable (agents no longer cite three-month-old workflow flags), and the agents have been easier to debug (you can read the structured memory; you can't read embedding vectors).
If you're starting an agent project today, I'd recommend skipping the vector-DB-first reflex and starting with structured KV + an MCP memory layer. Add vector retrieval when you have a specific workload that justifies it, not as the default architecture.
The default has been wrong for a while. The conference talks haven't caught up.
If you want to see the structured-memory side concretely, the Claude Code memory demo shows the install / record / recall / lineage loop in four steps. If you only need plain key→value lookup without MCP semantics (cache, feature flags, session keys), a generic KV like basekv is enough.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。