This article was originally published in the ANTISOC Issue (Continuous Penetration Testing) of our free infosec zine, PROMPT#. Find it free online HERE or order your $3 physical copy on the Spearphish General Store.

In the ever-evolving world of cybersecurity, staying ahead of the curve is not just a goal—it’s a necessity. As new vulnerabilities emerge, the race to identify and mitigate them begins. But how do we, the guardians of the digital realm, rapidly pinpoint these threats as they become public? Let’s dive into the fascinating world of vulnerability identification and see how the magic happens.

Past Lessons Can Fuel New Ideas

Seasoned red teamers know that every engagement leaves behind a trail of lessons. It’s not just about pride or curiosity—there’s a real operational need to find alternative methods. If you can’t stealthily deploy your favorite technique, or if you need a new exploit to bypass a patched vulnerability, you must adapt.

By discovering fresh weaknesses or writing custom exploits, red teams gain an edge, ensuring they can continue to test their targets effectively and raise the bar for an organization’s defenses. The first stop is often the collective wisdom of the security community. Researching existing vulnerabilities, blog posts, or advisories related to your target technology can help you understand the current landscape and locate opportunities to investigate.

The Mindset: Curiosity, Tenacity, and a Basement Full of Tools

Many aspiring security professionals who dream of tapping into malware and exploit development envision a hacker hunched over a keyboard in a dimly lit basement, running IDA Pro or Ghidra late into the wee hours of the night. While the caricature might be extreme, it’s not that far from reality for dedicated reverse engineers. And yet, it’s crucial to emphasize that discovering exploits isn’t some unattainable black magic. All it takes is patience, a hefty dose of curiosity, and a willingness to experiment. It really boils down to three things:

Patience: Identifying a vulnerability might mean combing through hundreds of functions or debugging a complex crash scenario dozens of times.

Curiosity: You have to want to understand how something works (and breaks) at a deeper level, beyond the obvious functionality.

Experimentation: Break things. Step through code. Try weird inputs. You might be surprised at what you discover.

For new red teamers, the biggest hurdle is often psychological. Reverse engineering with tools like IDA Pro, Ghidra, OllyDbg, or Radare2 may seem daunting at first, but there are tons of great resources on the internet that will help you get started. There are also freely available non-reverse engineering tools such as System Informer, Process Monitor, API Monitor, and more. Once you know your way around these interfaces and build a methodical process, they become powerful allies in your hunt for exploits.

Don’t believe me? Well, during one of our After-Action Reviews (AARs), the conversation came up that our bag of tricks was lacking initial access droppers and persistence. So, I began by studying what was currently being used and what was public. This search clued me in to some of the most recent techniques. I then looked at what was getting us caught.

I started looking at my research virtual machine (which I always recommend you have) and used Process Explorer to examine applications. I then ran Process Monitor over and over again, watching every event and digging into events I was not familiar with. (For example: I noticed a lot of queries to the same area of the registry that were returning different results. Since I did not recognize these calls, I scrutinized them further.)

It sounds tedious, but it works. This is how I was able to discover several key deficiencies that led to the research that eventually created the tool FaceDancer (https://www.blackhillsinfosec.com/a-different-take-on-dll-hijacking/).

So, the next time you hit a wall during a red team engagement—be it a failed payload, an uncrackable privilege escalation, or a non-persistent backdoor—remember that this challenge is often the spark that leads to major breakthroughs. Reverse engineering isn’t some dark art reserved for a few geniuses in poorly lit basements; it’s a practical discipline, grounded in patience, curiosity, and a willingness to learn from each new lead.

By defining the specific need (the “why?”), conducting thorough research, systematically reverse-engineering potential targets, and rigorously testing new exploits, red teams continually push the envelope. This is how they help organizations harden their defenses, educate security professionals, and keep the digital world a little bit safer from the threats that lurk around every corner.