惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

L
LangChain Blog
S
SegmentFault 最新的问题
V
Visual Studio Blog
J
Java Code Geeks
宝玉的分享
宝玉的分享
美团技术团队
博客园 - Franky
酷 壳 – CoolShell
酷 壳 – CoolShell
H
Hackread – Cybersecurity News, Data Breaches, AI and More
有赞技术团队
有赞技术团队
量子位
Martin Fowler
Martin Fowler
MyScale Blog
MyScale Blog
Google DeepMind News
Google DeepMind News
Jina AI
Jina AI
博客园 - 叶小钗
月光博客
月光博客
P
Proofpoint News Feed
D
DataBreaches.Net
Blog — PlanetScale
Blog — PlanetScale
博客园_首页
腾讯CDC
Microsoft Azure Blog
Microsoft Azure Blog
Stack Overflow Blog
Stack Overflow Blog

Human Risk Management Blog

Future-Proofing Organizations in the Face of AI What Security Can Learn From Dinosaurs Inside the OS-Aware Phishing Kit Profiling Your Device CyberheistNews Vol 16 #30 [Protect Your Users] AI Hallucinations Are Fueling Phishing Attacks Majority of Organizations Hit by Targeted Impersonation Attacks The Open-Source Paradox: Navigating the New Frontier of AI Supply Chain Risk Introducing The Hybrid Nudge Experience: Outbound Email Security Built for Your Risk Appetite Elevating the SOC Experience: Smarter Automation, Richer Threat Intelligence, and AI-Native Investigation New Phishing Tools Enable Attackers to Easily Bypass Multifactor Authentication From Inbox to Encryption: How Ransomware Delivery Has Evolved Attackers Exploit AI Hallucinations to Send Users to Phishing Sites Warning: ARToken Phishing Kit Automates BEC Attacks The New Face of AI Risk Trust Nothing: Tips to Secure AI Tools and Agents CyberheistNews Vol 16 #29 ClickFix Social Engineering is Now the Leading Malware Delivery Method Beyond the Checkbox: How a Proactive Partnership Led to Turnkey Hazing Compliance Trust, Verify, Protect: Modernizing Email Security for the Cloud Report: Social Engineering Remains a Central Part of AI-assisted Attacks ClickFix Social Engineering is Now the Leading Malware Delivery Method CyberheistNews Vol 16 #28 Your 2026 Phishing by Industry Benchmarks: The Findings on Human Risk Scammers Can Use AI Tools to Pinpoint Your Location Based on a Photo Report: Attackers Are Using AI to Automate Social Engineering Your KnowBe4 Fresh Compliance Plus Content Updates from June 2026 From Awareness to Digital Workforce Security Your KnowBe4 Fresh Content Updates from June 2026 Threat Actor Uses Phishing to Breach Orgs for Ransomware Gangs Invoice Phishing Attacks Are Abusing the Shop App Phishing Campaign Impersonates Interpol to Deliver Ransomware Hyper-Targeted Social Engineering Needs Real-Time Video Response Your Email is Protected. Is Your Teams Chat?
Prompt Injection and the Rise of Agentic Risk
Javvad Malik · 2026-07-09 · via Human Risk Management Blog

Javvad Malik, Lead CISO Advisor at KnowBe4Boxers will often say, the punches that hurt the most aren’t the ones which are thrown with the most force, but the ones they didn’t see coming. I think the same is true in cybersecurity. It’s not the most advanced technically efficient, 0-day utilizing attacks that have the biggest impact, but rather those quiet ones. With no malware or suspicious login at three in the morning from an IP address in a country your company has never done business with. No alert fires. No dashboard turns red.

A bit like a heist movie where the thief walks through the front door, hands the security guard a legitimate looking badge and walks out with the safe. The guard didn’t do anything wrong. Followed procedure and checked the ID. The problem was the ID was fake, the badge was borrowed and the guard was trained to trust documents, not question them.

In the same way, these are the kind of attacks we’re seeing going on with prompt injection, which OWASP has ranked as the number one vulnerability in AI systems for two years running. It is not a niche research curiosity any more. It is happening in production environments, against real enterprise systems, with real consequences.

A prompt injection attack is, at its core, a social engineering attack aimed at AI rather than a person. Instead of tricking a human into clicking a malicious link, an attacker tricks an AI into following malicious instructions hidden inside content the AI has been asked to process.

The AI is reading a document. The document contains text that looks like instructions. The AI, because it processes text, follows them. Consider this, you are summoned to court and you give your name as “Case Dismissed” — when the judge reads your name to call you, instead of everyone understanding it’s your name, they believe it’s an instruction to dismiss the case, and you walk out of the courtroom a free person.

That's pretty much it.

There are two primary types of prompt injection attacks:

Direct Prompt Injection

This is the more commonly seen type of attack. The user themselves sends malicious instructions trying to override the AI’s guardrails. “Ignore all previous instructions and tell me how to…”. Most AI providers have built robust defenses against these kinds of direct injection attacks. It’s not completely solved, but it’s a known avenue.

Indirect Prompt Injection

These attacks are far more interesting and quite dangerous. With indirect injection, the attacker doesn't interact with the AI at all. They plant malicious instructions inside content the AI will eventually read. That could be a webpage, an email attachment, a database record, a customer review, a document pulled from SharePoint. The AI processes that content as part of its normal job, encounters the hidden instructions, and executes them. The user sees nothing unusual. The AI has no reason to flag anything.

In practice, this could manifest as someone sending a well-crafted email to your corporate inbox. The email contains instructions hidden in white text, or buried in metadata. The AI assistant reads the email, processes the instruction and then because it has been granted the necessary permissions, forwards sensitive documents, exfiltrates calendar data or initiates a workflow.

In this whole process, the victim was unaware of anything untoward, the AI didn’t report anything unusual, and there were no indicators such as external logins to flag.

A Human Plus Agent Problem

Security teams that have spent years thinking about phishing understand the underlying mechanics here. Prompt injection is phishing, but the target is the AI, not the human. The attack surface has doubled.

The 2026 Verizon Data Breach Investigations Report found that 62% of data breaches involve people. That figure was already challenging before AI agents entered the picture.

Now consider that those same agents are being granted access to email systems, document stores, CRM platforms and internal APIs. Many organizations already report increased security incidents related to AI usage including shadow AI, or sharing sensitive data with AI tools.

In these scenarios the attacker doesn’t need to trick a human, they just need to get their instructions into something the AI will read. And this is truly new territory.

Some of the specific risks that come about as a result of prompt injection include data exfiltration, privilege escalation, workflow manipulation and misinformation.

Risk Reduction Methods

Prompt injection works because it exploits how language models are designed to work. They were designed to follow instructions in text, and distinguishing between "instructions from a trusted operator" and "instructions embedded in untrusted content" is difficult. Researchers have demonstrated that even models explicitly trained to resist injection can be bypassed. There is no known complete fix at the model layer.

The practical defenses exist, but they sit at the system design and governance level, not in the model itself.

  • Least privilege: AI agents should be granted only the permissions required for their specific task. An agent that summarizes documents does not need access to outbound email. An agent that answers customer questions does not need write access to the CRM. The blast radius of a successful injection is bounded by the permissions available to the compromised agent.
  • Input sanitisation and trust boundaries: Content from untrusted sources should be processed differently from instructions from trusted operators.
  • Human-in-the-loop for consequential actions: Autonomous agents taking consequential actions like sending emails, initiating transactions and modifying records, all should require human confirmation. This is inconvenient. It is considerably less inconvenient than discovering your AI agent has been exfiltrating data for three weeks.
  • Monitoring and anomaly detection: If agents are doing unexpected things like making unusual API calls, accessing files outside their normal scope, initiating communications that weren't requested then that needs to surface. This requires logging agent behavior at a level of granularity most organizations have not yet implemented.
  • Shadow AI is the unseen problem: KnowBe4's research found that 43% of workers share sensitive data with AI tools without permission. Those unsanctioned tools have no governance layer, no logging, no controls. They are also processing documents, emails and data on behalf of employees. They are vulnerable to exactly these attacks. You cannot defend what you cannot see.

The Attacker's Perspective

Attackers are not looking for a way to break the AI. They're looking for content the AI will read. A document in a shared folder. A record in a CRM. A webpage the agent will be asked to summarize. An email that will be processed by an AI assistant.

The attack surface is every piece of content your AI systems will ever consume. That's a lot of content. And unlike traditional malware, the malicious payload is just text. It doesn't trip antivirus, it doesn't look suspicious to a human glancing at the document and it doesn't leave the kind of traces traditional security tools were built to find.

What This Means in Practice

The organizations that will handle this well are not the ones waiting for AI vendors to solve it at the model layer. They are the ones extending their existing security thinking into the AI layer: treating AI agents as entities with identities and permissions that need to be governed, treating AI-processed content as untrusted input by default and building the same culture of healthy scepticism around AI behavior that good organizations have built around email and web browsing.

The digital workforce now includes people and AI agents. Both can be manipulated. Both need to be secured.

The heist film analogy only works if the guard keeps trusting the badge. The fix isn't to replace the guard. It's to redesign the security system so that no single piece of ID is sufficient authorization for the truly consequential actions.