The siren song of automation has been an old tune for security marketers for the better part of two decades at this point. Security vendors have promised that automation will solve the security skills gap. That it will make it possible for SOC teams to do more with fewer people. That it will revolutionize security analyst workflows. That it will reduce alert fatigue and solve analyst burnout.

And yet, here we are in 2020 with plenty of security teams still battling false positives and false negatives the same as always and disabling more automated workflows than ever. So what gives? Why hasn’t automation lived up to the promises of the past?

One of the big problems with effective automation is the current threat landscape heavily complicating analysis and response to threats. Many years ago, when automation was being touted as the fix for strained or ineffective SOCs, straying from the standard workflow wasn’t a regular occurrence. However, as the threat landscape has evolved, so have our defenses, forcing attackers to innovate and overcome. This means many AI models, machine learning models and workflows that drive automation have to be reimagined, not just improved.

Back in the first waves of security automation investment and hype, we would see the same attack, with the same indicators and schema, or a similar-looking domain, maybe with a character changed here and there to beat simple detection signatures. I remember several instances where an APT actor would use the same IP for years. In that environment, AI-backed automation and consistent response workflows would have enough stable data to start to form very good models for accurate and effective automation. That simplicity is unheard of now.

WHERE WE REALLY STAND WITH AUTOMATION

Techniques are changing constantly, and more than just a simple character shift or slight edit. The ability of models to detect these constant shifts is not quite there yet. And trying to feed all of that into a model is exhausting and difficult. Which means that the human still has to be a big part of the decision-making process because we’re still the ones that can reliably and effectively make that decision of whether something’s a true positive or false positive, and keep a sharp look out for the false negatives where the system doesn’t even alert because the model is not able to detect the latest update or version of an attack or malware.

Now, this isn’t to say that automation has no role in the SOC. Certain kinds of automation can be extremely beneficial for security analysts, but we have to think more of micro-automations. Thinking about small parts of the overall workflow, such data collection, which is typically the number one time suck for an analyst today, can have a profound effect on your operation. When an analyst gets a user-based alert that they need to run down, automation can be invaluable to collect information from all the right sources and present it in a unified manner—which can sometimes take as much as 20 minutes to do for every initial investigation that needs to be made. Thinking smaller, enables more human analysis, validation and verification before moving to mitigation and response. Once you start getting past triage, the case for across-the-board automation gets a lot trickier.

A lot of companies have touted automated triage for years, but the problem is there are many situations in real-world organizations where automated containment, for example, is not going to fly. If you have an automated system isolate the machine of a VIP or critical system based on an alert without an analyst taking a look first, it can cause big problems. Although many systems can take this into account, the inclusion of the human can prevent costly mistakes where automation does too much. This is just one example of many where human intervention and decision-making breakpoints need to be included in the process.

AUTOMATION AS A TOOL

The point is, automation is a tool for analysts and not vice versa. Analysts shouldn’t be contorting themselves to make automation work—if a piece of security automation adds more problems and work into the triage and response process than it solves, then it probably isn’t good automation in the first place. As an industry, we must tread carefully as we figure out the right level of automation and human intervention that makes sense for meaningfully reducing risk to the enterprise

Cyborg is all about supporting today’s security analysts with pertinent information that isn’t just fluff. Continue reading how in our post Network Content and You: Why Logs Matter in the Age of TLS/SSL.