Aug 07 2025
Security

Black Hat 2025: Penetration Testing Evolves With AI Capabilities

Hackers are using artificial intelligence to work faster and more efficiently. Ongoing pen testing enables small businesses, such as defense contractors, to remediate vulnerabilities proactively.

Artificial intelligence is a valuable tool that makes it easier to work faster, smarter and more efficiently. Unfortunately, the same is true for cybercriminals.

At this year’s Black Hat USA conference in Las Vegas, experts shared some of the specific ways threat actors are using AI to become faster and more sophisticated, making them more dangerous to organizations, especially small businesses.

“Their favorite initial access vectors remain simply exploiting internet-facing, publicly known, unpatched vulnerabilities,” said Bailey Bickley, chief of defense industrial base (DIB) defense at the National Security Agency. “They are getting really good at using AI to find and exploit unpatched instances of these vulnerabilities at scale.”

If that weren’t enough risk, these unpatched vulnerabilities aren’t the only pathway cybercriminals are exploiting. They’re also using AI to steal users’ credentials. “Attackers don’t have to hack in; they’ll log in,” said Snehal Antani, CEO of Horizon3.ai.

“Most of the tactics to compromise those credentials didn’t require Common Vulnerabilities and Exposures,” Antani said of a red team test performed by his company. As a result of the test, 20% of the initial credentials the company compromised were domain administrator credentials, “which means we got keys to the kingdom almost immediately.”

Click the banner below to implement security insights from Black Hat in your organization.

 

Thinking like these cybercriminals is the first step to protecting your environment, he said. “In cybersecurity, the only perspective that matters is the attacker’s perspective. What does your environment look like through the eyes of the attacker, and how do you use that perspective to fix problems that matter?”

What’s New for Penetration Testing?

Because the cybercriminals are armed with AI, businesses need solutions of the same caliber to defend their environments.

“We need to use AI and automation first, fast and for defense,” Bickley said.

“The whole goal here is that offense drives defense,” Antani noted. “Offense helps make sure you’re facing problems that matter.”

AI helps organizations defend their environments at scale, matching the speed and efficiencies of threat actors, even when IT departments are comparatively stretched thin.

The penetration testing process previously took a long time: IT needed to first get the expenditure approved, then work with a team of security experts who poked and prodded business’s network defenses.

When organizations used Horizon3.ai’s NodeZero platform, Antani noticed “a shift toward continuously assessing your environment, fixing problems that actually mattered, and quickly running a retest to verify that you’re good to go.”

DIVE DEEPER: Optimize cyberdefense with managed security services.

However, finding the problems that actually mattered and — more specifically — “deciding what not to fix” were always challenges, Antani said.

AI offers solutions for that too.

Weighing a pen test’s value by its ability to find problems is a legacy way of thinking, Antani told Black Hat USA attendees. “The goal of the pen test is to fix problems that matter,” he said.

An automated pen test can make these identifications as part of its assessment. “Now, suddenly, what’s exploitable is what you’re going to go off and prioritize,” Antani said.

Snehal Antani
In cybersecurity, the only perspective that matters is the attacker’s perspective.”

Snehal Antani CEO, Horizon3.ai

NSA’s Defense Industrial Base Comprises Small Businesses

The DIB is 80% small businesses, Bickley said. “These are companies that have outsourced IT, minimal IT staff and minimal awareness of security best practices.”

As an example, she talked about a company that makes custom radio-frequency solutions for the Department of Defense: “Although they are an incredible manufacturer, their IT environment was not quite what I had had in mind for a defense contractor. These are the companies facing off against nation-state-backed actors in what has been a fundamentally unfair fight.”

The DIB is not made up of traditional defense contractors today, Bickley explained. Instead, it includes:

  • AI companies whose models are being adopted for use in DOD applications
  • Commercial transportation companies that don’t consider the DOD their primary customer, but that can be called upon to move aid in and out of war zones in times of conflict or crisis
  • Foreign-owned water, gas and telecommunication companies that support all of our military bases overseas

“The battle space is really changing,” Bickley said, adding that DOD and NSA recognized a need for new, scalable solutions capable of covering all of this new ground.

Success With Continuous Automated Penetration Testing

Looking for a security solution that could meet the needs of the DIB’s small businesses, the NSA’s Cybersecurity Collaboration Center worked with Horizon3.ai to offer NodeZero to 200 defense contractors.

“We saw those companies conduct over 20,000 hours of pen testing, because we were able to make it automated and accessible for them,” Bickley said. As a result, the companies found 50,000 vulnerabilities and mitigated 70% of them.

In one example, a research and development company joined DIB and had multiple DOD contracts. When it ran a pen test against its internal networks, within five minutes NodeZero accessed a file share with more than 3 million files. Within those files was sensitive information related to nuclear-powered submarines and aircraft carriers.

Automation also helps NSA flag potential vulnerabilities for contractors when they become known. “They’re not thinking about 2-year-old vulnerabilities,” Bickley said of the small businesses. “We are able to share insights on what we’re seeing in the threat environment and flag things for these companies so they can stay on top of it.”

Keep this page bookmarked for articles from the event, and follow event highlights and behind-the-scenes moments on the social platform X @BizTechMagazine and @BlackHatEvents.

Douglas Rissing/Getty Images
Close

See How Your Peers Are Leveling Up Their IT

Sign up for our financial services newsletter and get the latest insights and expert tips.