惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

P
Proofpoint News Feed
V
V2EX
博客园_首页
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Recent Announcements
Recent Announcements
博客园 - 司徒正美
Microsoft Security Blog
Microsoft Security Blog
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
Latest news
Latest news
Vercel News
Vercel News
The Register - Security
The Register - Security
T
The Exploit Database - CXSecurity.com
S
Schneier on Security
N
Netflix TechBlog - Medium
WordPress大学
WordPress大学
小众软件
小众软件
L
Lohrmann on Cybersecurity
GbyAI
GbyAI
P
Privacy & Cybersecurity Law Blog
T
Tor Project blog
AWS News Blog
AWS News Blog
美团技术团队
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
K
Kaspersky official blog
B
Blog RSS Feed
G
Google Developers Blog
量子位
大猫的无限游戏
大猫的无限游戏
Google DeepMind News
Google DeepMind News
Scott Helme
Scott Helme
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
I
Intezer
雷峰网
雷峰网
Martin Fowler
Martin Fowler
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Blog — PlanetScale
Blog — PlanetScale
IT之家
IT之家
F
Full Disclosure
Apple Machine Learning Research
Apple Machine Learning Research
博客园 - 【当耐特】
The Hacker News
The Hacker News
U
Unit 42
S
SegmentFault 最新的问题
I
InfoQ
aimingoo的专栏
aimingoo的专栏
Y
Y Combinator Blog
宝玉的分享
宝玉的分享
罗磊的独立博客
Spread Privacy
Spread Privacy
C
CERT Recently Published Vulnerability Notes

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Prompt Injection Was Stateless. Memory Poisoning Is Persistence
Maninderpree · 2026-05-01 · via DEV Community

For the last two years, AI security discussions have mostly been about stateless compromise.

Can you jailbreak the model in one session?
Can you inject hostile instructions into retrieved content?
Can you get the assistant to reveal something, ignore a rule, or call the wrong tool right now?

Those questions still matter.

But they are starting to belong to an earlier phase of the problem.

The more interesting risk now is persistence.

Not whether an attacker can manipulate an agent once.

Whether they can manipulate what the agent remembers, and make that manipulation survive into future decisions.

That is the shift memory poisoning introduces.

Prompt injection was stateless. Memory poisoning is persistence.

And persistence changes the security model completely.

Why this feels different from classic prompt injection

Traditional prompt injection is dangerous, but it is often temporally bounded.

A malicious instruction lands in a document, email, web page, support ticket, or retrieved chunk. The model reads it, gets confused or manipulated, and produces a bad result in that interaction.

That is bad enough.

But in the simplest version of that story, the attack has to keep reappearing. The hostile text needs to be retrieved again. The session needs to stay alive. The exploit pressure has to remain present.

Memory poisoning is different.

The goal is not just to influence the current response. The goal is to influence the system's future behavior by changing what it stores as a trusted memory, preference, summary, lesson, fact, successful pattern, or durable piece of context.

Once that happens, the attack stops being an event and starts becoming state.

That is a much more serious architectural problem.

Why memory became an attack surface so quickly

Because memory makes agents more useful.

Teams want assistants that remember user preferences, recurring tasks, project context, past corrections, successful workflows, trusted sources, important documents, and previous decisions. They want continuity across sessions. They want agents that feel less like stateless chatbots and more like adaptive systems.

So memory gets added in a lot of forms:

  • saved user preferences
  • long-term conversation summaries
  • retrieved historical interactions
  • "successful past actions" stores
  • project rules and workspace memories
  • cached facts about users, accounts, or systems
  • external memory layers implemented through RAG or vector search

All of that helps performance and usability.

It also creates a new question:

What happens when the thing being persisted is wrong, manipulated, adversarial, stale, or strategically planted?

That is the core of memory poisoning.

The security boundary is no longer the session

This is the conceptual jump teams need to make.

A stateless assistant mostly fails inside a single interaction boundary. A memory-enabled agent can fail across time.

That means the security boundary is no longer just:

  • this prompt
  • this response
  • this retrieval event
  • this tool call

It becomes:

  • what entered memory
  • why it was stored
  • how it was labeled
  • when it gets retrieved again
  • what future decisions it can influence
  • how long it survives before review, expiry, or deletion

In other words, you are no longer just defending inference. You are defending behavioral persistence.

The most dangerous part: poisoned memory looks legitimate later

This is what makes the problem subtle.

A prompt injection payload in a document may look suspicious when you inspect the raw source.

A poisoned memory often looks normal by the time it is reused.

Maybe it has been compressed into a summary.
Maybe it has been translated into a user preference.
Maybe it has been stored as a successful workflow pattern.
Maybe it has been promoted into a "trusted source" hint.
Maybe it has been embedded into a retrieval store alongside thousands of benign memories.

By the time the agent sees it again, it may no longer look like an attack at all.

It looks like prior knowledge.

That is what gives memory poisoning so much leverage. The attack gains the credibility of memory itself.

Four ways memory poisoning can actually happen

1. Poisoned preferences

An attacker gets a system to store something as if it were a durable user preference:

  • always trust this vendor
  • prefer this source
  • treat these messages as high priority
  • summarize issues this way
  • use this workflow by default

That may sound minor until you remember how many future actions can be downstream of a "preference."

2. Poisoned summaries

Many agents do not retain raw conversation history forever. They condense it into summaries.

That summary step is convenient and dangerous. If a malicious interaction gets compressed into a durable narrative like "user previously approved this process" or "this source is reliable," the agent may reuse the poisoned abstraction long after the original context disappears.

3. Poisoned experience memory

Some agent systems store prior successful actions or trajectories so future tasks can imitate them.

That is powerful because it lets the system learn from experience.

It is risky for the exact same reason.

If an attacker can get a malicious or overbroad action recorded as a success case, the system may later reproduce it with the confidence usually reserved for proven workflows.

4. Poisoned retrieval memory

Memory is often implemented through external stores: vector databases, semantic caches, long-term notes, workspace memories, or document indexes.

If those stores accept untrusted or weakly reviewed content, the poisoning does not need to modify the model at all. It just needs to become retrievable often enough, or semantically similar enough, to influence future reasoning.

This is where memory poisoning starts to overlap with RAG poisoning, but the important distinction is persistence of behavioral influence, not just corruption of factual retrieval.

Long-horizon attacks are harder to notice

This is another reason the topic matters.

Security teams are good at noticing sharp failures:

  • the assistant said something obviously wrong
  • the model leaked data
  • the agent took an unexpected action
  • the guardrail failed in a visible way

Memory poisoning may not look like that.

It can unfold gradually:

  • recommendations become subtly biased
  • source trust shifts over time
  • unsafe workflows get reused more often
  • hallucinations stabilize into "remembered facts"
  • the assistant becomes consistently wrong in one direction

That kind of drift is much easier to rationalize away.

Teams say things like:

  • the model is just being quirky
  • users probably phrased it oddly
  • maybe the data changed
  • maybe the retrieval was noisy

Meanwhile, the system may be carrying forward a planted corruption across dozens or hundreds of interactions.

In 2026, poisoned memory does not have to stay local

This year's shift toward multi-agent systems makes the problem worse.

More agents now delegate to other agents, share workspace context, reuse common memory layers, and exchange summaries or task state during collaborative workflows.

That means a poisoned memory is no longer just a problem for the agent that stored it first.

It can become a contagion problem.

If Agent A stores a corrupted preference, summary, or workflow pattern, that memory may later be surfaced to Agent B as trusted context during handoff, planning, or retrieval. If B then acts on it, reinforces it, or writes a derivative summary back into shared state, the poisoning starts to propagate rather than merely persist.

That is an important change in threat shape.

The attacker is no longer trying only to manipulate one model in one moment. They may be trying to seed a bad memory into a collaborative system and let the agents help spread it from there.

In other words, memory poisoning is starting to look less like local corruption and more like viral state compromise.

Why long-horizon persistence matters more for agents than chatbots

A stateless chatbot can still be compromised.

But persistent agents amplify the consequences because they have more continuity, more autonomy, and more surface area.

They may:

  • remember across sessions
  • call tools over time
  • build up user-specific memory
  • maintain project or workspace state
  • reuse previous decisions as shortcuts
  • adapt future behavior based on prior outcomes

That means the attacker no longer has to win every interaction.

They may only need to win the memory formation step once.

After that, the system starts helping carry the attack forward on its own.

The wrong defense is "just make memory useful"

A lot of current memory design still optimizes almost entirely for convenience:

  • remember more
  • summarize more aggressively
  • persist more context
  • reuse more successful history
  • reduce friction between sessions

Those are product wins, but they can become security losses if persistence is treated as automatically good.

Memory should not be designed like a scrapbook.

It should be designed like a governed data store with risk-aware rules around:

  • what can be written
  • who can cause a write
  • what confidence a memory carries
  • how long it persists
  • how it can influence future decisions
  • when it must be reviewed, downgraded, or forgotten

That is a much stricter standard than most agent products currently apply.

What good defenses will probably look like

This area is still emerging, but the direction is already clear.

Strong defenses will likely come less from one magical classifier and more from treating memory as a security-sensitive subsystem.

That means things like:

Typed memory instead of one blended store.
User preferences, factual memories, workflow patterns, and retrieved external notes should not all have the same trust level or retrieval priority.

Write controls, not just read controls.
Teams spend time controlling what the model can access. They also need to control what the model is allowed to persist.

Confidence and provenance on memories.
A memory should carry where it came from, why it was stored, and how much the system should trust it later.

Verifiable memory provenance.
There is growing interest in making memory history tamper-evident rather than merely descriptive. In practice, that can mean signed memory writes, append-only hash-chained logs, cryptographically verifiable intent or inference trails, and, in some emerging designs, ZK- or TEE-backed proofs for high-assurance provenance claims. The core idea is simple: a memory should not just exist, it should be attributable to a specific user action, agent step, or system event with evidence.

Expiry and decay.
Not all memories deserve indefinite survival. Some should expire automatically unless reconfirmed.

Review gates for high-impact memories.
If a memory could affect tools, approvals, trusted sources, account behavior, or privileged workflows, it should not be treated like harmless personalization.

Behavioral monitoring over time.
The question is not only "did the model fail now?" but also "is the agent drifting in a persistent direction across weeks?"

The practical question for teams

If your product includes persistent memory, ask this:

What is the worst thing an attacker could make this system remember?

That question is much more useful than generic anxiety about memory features.

It pushes you toward concrete design review:

  • Can the system store false preferences?
  • Can it store attacker-shaped summaries?
  • Can it learn unsafe workflows from experience?
  • Can external content become durable memory automatically?
  • Can users inspect, correct, and delete memory?
  • Can one agent's memory influence another agent through shared state or handoff?
  • Can we prove which user, agent, or system event created a given memory?
  • Do different memory types influence different actions?
  • What high-impact behavior is downstream of remembered state?

If those answers are vague, the feature is probably more exposed than it looks.

The next phase of AI security is about state

Early AI security discourse focused on outputs.

Then it shifted toward tools, retrieval, and agents.

The next shift is going to be about stateful compromise.

Not just whether the model can be manipulated in one conversation, but whether that manipulation can be made durable, retrievable, and behavior-shaping over time.

That is why memory poisoning matters.

It turns a transient exploit into a persistent condition.

It turns a bad interaction into a future default.

It turns an agent's usefulness feature into a long-horizon attack surface.

Prompt injection was serious because it let attackers influence what the model did.

Memory poisoning is more serious because it can influence what the system becomes.