Tom Kellermann, VP of AI Security and Threat Intelligence at TrendAI.
getty
AI has become the new foundation of the modern software stack, and at the core of AI lies intelligence derived from data.
AI often has access to a myriad of tools and systems. But that access can be easily abused. AI can bring with it a sentient, malicious and dark passenger. LLMs can be poisoned and forced to hallucinate, or they can also be jailbroken and used for nefarious purposes.
As I've observed in the industry, many cybercrime cartels are starting to automate their efforts, employing purpose-built and legitimate AI platforms to weaponize and enhance their operations. Fraud campaigns like phishing can now be run continuously.
At my company, we've observed five key, ominous trends:
1. Supply chains can be weaponized. We've observed a 35% surge in AI-related common vulnerabilities and exposures (CVEs).
2. We've recently seen the arrival of the first dynamically generated malware families.
3. Deepfake technology has exploded, leading to crimes ranging from kidnapping to business email compromise.
4. We've seen the resurgence of steganography, the process in which malware is hidden in image or video files.
5. We've seen jailbreak-as-a-service providers gain notoriety in the industry.
In the AI era, security cannot be an afterthought; it must be part of your design from the beginning. This must include continuous exposure management, identity context, telemetry correlation, human-in-the-loop controls and the ability to validate and simulate adversary paths before they are exploited.
Offense must inform your defense. As such, CISOs must predict, prevent, detect and respond to threats faster.
To do this, AI can be a valuable tool. It can enhance threat detection and response. AI can monitor networks and systems in real time, analyzing the abnormal behavior patterns of both employees and other users. It can detect and prioritize potential threats like malware, then automate security tasks, such as incident response protocols and more. This can then free up cybersecurity personnel to focus more strategically.
All of that said, AI must be secured in order for it to function optimally. To do this, your organization must adhere to OWASP and NIST guidelines.
In addition, when implementing AI, there are a few best practices that I use on a daily basis:
1. Be sure to develop input validation and context isolation systems.
2. Don't blindly trust prompts. Deploy input filters, output validators, access controls, monitoring and incident response capabilities.
3. Verify responses before users or systems act on them.
4. Restrict what the LLM can do. Employ least-privilege controls as a baseline.
5. Conduct red-teaming tests and event monitoring. Test continuously and flag problems like data leakage, tool misuse and resource abuse.
Threat detection and response must be accelerated across your entire digital estate. This will help reduce your attack surface and bolster your defenses against AI-powered phishing.
As vulnerability accelerates, our real challenge will become prioritization and remediation at scale. Without that, increased discovery will only compound your security debt. That's why vulnerability shielding may be your best bet in the face of this new cybersecurity normal.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。