惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

人人都是产品经理
人人都是产品经理
MyScale Blog
MyScale Blog
Y
Y Combinator Blog
罗磊的独立博客
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
P
Proofpoint News Feed
Google DeepMind News
Google DeepMind News
V
Vulnerabilities – Threatpost
T
The Blog of Author Tim Ferriss
云风的 BLOG
云风的 BLOG
Recorded Future
Recorded Future
N
News and Events Feed by Topic
B
Blog RSS Feed
阮一峰的网络日志
阮一峰的网络日志
博客园_首页
C
CXSECURITY Database RSS Feed - CXSecurity.com
博客园 - 【当耐特】
N
Netflix TechBlog - Medium
博客园 - 叶小钗
B
Blog
Vercel News
Vercel News
T
Tenable Blog
T
The Exploit Database - CXSecurity.com
Spread Privacy
Spread Privacy
T
Threat Research - Cisco Blogs
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Last Week in AI
Last Week in AI
F
Fortinet All Blogs
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
Microsoft Security Blog
Microsoft Security Blog
S
Securelist
Microsoft Azure Blog
Microsoft Azure Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
P
Palo Alto Networks Blog
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
D
DataBreaches.Net
Cyberwarzone
Cyberwarzone
Engineering at Meta
Engineering at Meta
Martin Fowler
Martin Fowler
G
GRAHAM CLULEY
Project Zero
Project Zero
Cisco Talos Blog
Cisco Talos Blog
A
Arctic Wolf
C
CERT Recently Published Vulnerability Notes
L
LangChain Blog
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
C
Check Point Blog
A
About on SuperTechFans
W
WeLiveSecurity
The GitHub Blog
The GitHub Blog

Human Risk Management Blog

Trust, Verify, Protect: Modernizing Email Security for the Cloud Report: Social Engineering Remains a Central Part of AI-assisted Attacks ClickFix Social Engineering is Now the Leading Malware Delivery Method CyberheistNews Vol 16 #28 Your 2026 Phishing by Industry Benchmarks: The Findings on Human Risk Scammers Can Use AI Tools to Pinpoint Your Location Based on a Photo Report: Attackers Are Using AI to Automate Social Engineering Your KnowBe4 Fresh Compliance Plus Content Updates from June 2026 From Awareness to Digital Workforce Security Your KnowBe4 Fresh Content Updates from June 2026 Threat Actor Uses Phishing to Breach Orgs for Ransomware Gangs Invoice Phishing Attacks Are Abusing the Shop App Phishing Campaign Impersonates Interpol to Deliver Ransomware Hyper-Targeted Social Engineering Needs Real-Time Video Response Your Email is Protected. Is Your Teams Chat? CyberheistNews Vol 16 #27 [HOW TO] Your Cybersecurity Starts at Home on World Social Media Day 2026 Phishing by Industry Benchmarking Report: Findings on Human Risk Static DLP Is Leaving You in the Dark: Why It’s Time for Intelligent, Self-Serve Outbound DLP and Misdirected Content Analysis INC Ransomware Gang Targets the Legal Sector 5 Essential Cybersecurity Defenses for Cloud Email Security Cybercriminals Are Targeting the FIFA World Cup 2026 Why Bite-Sized Security Awareness Training Matters in an Age of TikTok and Digital Distraction Happy 3rd Birthday to Our KnowBe4 Community! Phishing Exposes Employee Data at 86% of Fortune 100 Companies Shadow AI Is Not Shadow IT With a Better Marketing Budget CyberheistNews Vol 16 #26 A New Extortion Scam Uses IT Impersonation to Breach Organizations Cybersecurity Starts At Home This World Social Media Day FTC Report: Americans Lost $3.5 Billion to Imposter Scams Last Year Report: Device Code Phishing is Surging Report: Online Shoppers Increasingly Ignore Scam Warning Signs Security Training Needs Google Maps, Not Christopher Columbus Turn Account Takeover Into Real-Time Security Coaching Extortion Gang Sends In-Person Attackers to Exfiltrate Data Attackers aren’t loyal to any collaboration channel CyberheistNews Vol 16 #25 [The AI Tell] How To Expose Machine-Written Phishing Fast Social Engineering Attacks Abuse Workplace Collaboration Tools New Extortion Brand Uses IT Impersonation to Breach Organizations APWG Report: Social Media Phishing is Surging Cybersecurity Awareness Training for AI: Key Focus Areas Americans Lost $900 Million to AI-Powered Scams Last Year What AI Can’t Hide When It Writes a Phishing Email Your AI Agents Are Eager to Please And Easy to Exploit From 1% to 26%: How AIDA Orchestration Fixes the Remedial Training Gap Best AI Agent Security Tools for SMB and Enterprise in 2026 4 Hot Summer Travel Tips To Avoid Scams CyberheistNews Vol 16 #24 [FBI Alert] Lock Down Your Microsoft 365 Device Code Flows Now The Role of Agentic AI in Phishing Security Training A Credit Score for Cyber Behavior Agentic AI Security in 2026: What to Know How to Secure AI Agents: 4 Best Practices An Overview of Email Compliance Regulations and Reporting Report: AI-Assisted Fraud is Surging Attackers Use Spoofed ChatGPT Site to Deliver Malware I Love Device-Bound Session Credentials, But They Are Still Phishable and Hackable Nearly Two-Thirds of CEOs Cite Cyberattacks as Their Top Concern A Look at Spam vs. Phishing: 4 Key Differences KnowBe4 Wins Multiple 2026 TrustRadius Top Rated Awards Cyber Insurance for Mid‑Market Organizations in Southeast Asia KnowBe4 Earns Multiple 2026 Buyer's Choice Awards from TrustRadius The New Frontier: Securing Japan’s Hybrid Digital Workforce (2026 & Beyond) CyberheistNews Vol 16 #23 Now Phishing Attacks Use Real Hotel Reservations to Target Travelers Report: AI-Enabled Social Engineering Attacks Are on the Rise Your KnowBe4 Fresh Compliance Plus Content Updates from May 2026 FBI: Kali365 Phishing Kit is Targeting Microsoft 365 Accounts KB4-CON - AI Is Everything How to Secure AI Adoption In Your Organization Your KnowBe4 Fresh Content Updates from May 2026 The Silent Invitation: A Deep Dive into Calendar Invite Phishing Cyber Insurance for Mid‑Market Organizations in Southeast Asia Chinese-Language Phishing Kits Are Growing More Advanced Phishing Attacks Are Using Real Hotel Reservation Info to Target Travelers Warning: Scammers are Exploiting Geopolitical Unrest Athletes Are Increasingly Targeted by Social Engineering Attacks AI Agent Governance Part 3 - Runtime Governance: The Hidden Performance Cost of Agentic AI AI Agent Governance Part 2 - What Good Looks Like: Governing AI Agents in Practice 8 Ways to Reduce False Positives in Email Security Ransomware Attacks Drive a Surge in Cyber Insurance Claims My Favorite 5 KnowBe4 Agents Perry Carpenter KB4-CON 2026 Q&A: Deepfakes & Deception Free Gift Fallacy: How Attackers Harvest Credit Cards via Fake Surveys When Global Conflict Becomes a Cyber Weapon: How Iran Tensions and Other Stressful Events Fuel Social Engineering Attacks CyberheistNews Vol 16 #21 [Heads Up] GitHub Breach Shows Developer Tools Are Social Engineering Targets Alert: Extortion Groups Are Using Phishing Kits to Automate Their Attacks Beyond the Chatbot: Why Your AI Agents are Your Newest (and Most Vulnerable) Colleagues Report: Adversarial Use of AI is Evolving
Prompt Injection and the Rise of Agentic Risk
Javvad Malik · 2026-07-09 · via Human Risk Management Blog

Javvad Malik, Lead CISO Advisor at KnowBe4Boxers will often say, the punches that hurt the most aren’t the ones which are thrown with the most force, but the ones they didn’t see coming. I think the same is true in cybersecurity. It’s not the most advanced technically efficient, 0-day utilizing attacks that have the biggest impact, but rather those quiet ones. With no malware or suspicious login at three in the morning from an IP address in a country your company has never done business with. No alert fires. No dashboard turns red.

A bit like a heist movie where the thief walks through the front door, hands the security guard a legitimate looking badge and walks out with the safe. The guard didn’t do anything wrong. Followed procedure and checked the ID. The problem was the ID was fake, the badge was borrowed and the guard was trained to trust documents, not question them.

In the same way, these are the kind of attacks we’re seeing going on with prompt injection, which OWASP has ranked as the number one vulnerability in AI systems for two years running. It is not a niche research curiosity any more. It is happening in production environments, against real enterprise systems, with real consequences.

A prompt injection attack is, at its core, a social engineering attack aimed at AI rather than a person. Instead of tricking a human into clicking a malicious link, an attacker tricks an AI into following malicious instructions hidden inside content the AI has been asked to process.

The AI is reading a document. The document contains text that looks like instructions. The AI, because it processes text, follows them. Consider this, you are summoned to court and you give your name as “Case Dismissed” — when the judge reads your name to call you, instead of everyone understanding it’s your name, they believe it’s an instruction to dismiss the case, and you walk out of the courtroom a free person.

That's pretty much it.

There are two primary types of prompt injection attacks:

Direct Prompt Injection

This is the more commonly seen type of attack. The user themselves sends malicious instructions trying to override the AI’s guardrails. “Ignore all previous instructions and tell me how to…”. Most AI providers have built robust defenses against these kinds of direct injection attacks. It’s not completely solved, but it’s a known avenue.

Indirect Prompt Injection

These attacks are far more interesting and quite dangerous. With indirect injection, the attacker doesn't interact with the AI at all. They plant malicious instructions inside content the AI will eventually read. That could be a webpage, an email attachment, a database record, a customer review, a document pulled from SharePoint. The AI processes that content as part of its normal job, encounters the hidden instructions, and executes them. The user sees nothing unusual. The AI has no reason to flag anything.

In practice, this could manifest as someone sending a well-crafted email to your corporate inbox. The email contains instructions hidden in white text, or buried in metadata. The AI assistant reads the email, processes the instruction and then because it has been granted the necessary permissions, forwards sensitive documents, exfiltrates calendar data or initiates a workflow.

In this whole process, the victim was unaware of anything untoward, the AI didn’t report anything unusual, and there were no indicators such as external logins to flag.

A Human Plus Agent Problem

Security teams that have spent years thinking about phishing understand the underlying mechanics here. Prompt injection is phishing, but the target is the AI, not the human. The attack surface has doubled.

The 2026 Verizon Data Breach Investigations Report found that 62% of data breaches involve people. That figure was already challenging before AI agents entered the picture.

Now consider that those same agents are being granted access to email systems, document stores, CRM platforms and internal APIs. Many organizations already report increased security incidents related to AI usage including shadow AI, or sharing sensitive data with AI tools.

In these scenarios the attacker doesn’t need to trick a human, they just need to get their instructions into something the AI will read. And this is truly new territory.

Some of the specific risks that come about as a result of prompt injection include data exfiltration, privilege escalation, workflow manipulation and misinformation.

Risk Reduction Methods

Prompt injection works because it exploits how language models are designed to work. They were designed to follow instructions in text, and distinguishing between "instructions from a trusted operator" and "instructions embedded in untrusted content" is difficult. Researchers have demonstrated that even models explicitly trained to resist injection can be bypassed. There is no known complete fix at the model layer.

The practical defenses exist, but they sit at the system design and governance level, not in the model itself.

  • Least privilege: AI agents should be granted only the permissions required for their specific task. An agent that summarizes documents does not need access to outbound email. An agent that answers customer questions does not need write access to the CRM. The blast radius of a successful injection is bounded by the permissions available to the compromised agent.
  • Input sanitisation and trust boundaries: Content from untrusted sources should be processed differently from instructions from trusted operators.
  • Human-in-the-loop for consequential actions: Autonomous agents taking consequential actions like sending emails, initiating transactions and modifying records, all should require human confirmation. This is inconvenient. It is considerably less inconvenient than discovering your AI agent has been exfiltrating data for three weeks.
  • Monitoring and anomaly detection: If agents are doing unexpected things like making unusual API calls, accessing files outside their normal scope, initiating communications that weren't requested then that needs to surface. This requires logging agent behavior at a level of granularity most organizations have not yet implemented.
  • Shadow AI is the unseen problem: KnowBe4's research found that 43% of workers share sensitive data with AI tools without permission. Those unsanctioned tools have no governance layer, no logging, no controls. They are also processing documents, emails and data on behalf of employees. They are vulnerable to exactly these attacks. You cannot defend what you cannot see.

The Attacker's Perspective

Attackers are not looking for a way to break the AI. They're looking for content the AI will read. A document in a shared folder. A record in a CRM. A webpage the agent will be asked to summarize. An email that will be processed by an AI assistant.

The attack surface is every piece of content your AI systems will ever consume. That's a lot of content. And unlike traditional malware, the malicious payload is just text. It doesn't trip antivirus, it doesn't look suspicious to a human glancing at the document and it doesn't leave the kind of traces traditional security tools were built to find.

What This Means in Practice

The organizations that will handle this well are not the ones waiting for AI vendors to solve it at the model layer. They are the ones extending their existing security thinking into the AI layer: treating AI agents as entities with identities and permissions that need to be governed, treating AI-processed content as untrusted input by default and building the same culture of healthy scepticism around AI behavior that good organizations have built around email and web browsing.

The digital workforce now includes people and AI agents. Both can be manipulated. Both need to be secured.

The heist film analogy only works if the guard keeps trusting the badge. The fix isn't to replace the guard. It's to redesign the security system so that no single piece of ID is sufficient authorization for the truly consequential actions.