Black Hat USA 2023: How Large Language Models Can Help Detect Phishing Attacks
Experts give an inside look at phishing attacks, what constitutes a effective phishing email and what tactics IT leaders should employ to outsmart them.
Lily is a Senior Editor at BizTech Magazine. She follows tech trends, thought leadership and data analytics.
“There are roughly 20,000 people at Black Hat right now. What if I create a phishing email that scams all of you?” said Fredrik Heiding, Ph.D research fellow at Harvard University. He smiled — he was joking, of course. But in this scenario, Heiding demonstrated just how easy it is now to write a successful phishing email using ChatGPT.
According to experts, phishing attacks need only a few accurate details and data points about their recipients to work. “If they are targeted to the right people at the right time — like, say, participants at a security conference with the offer of a free voucher for dinner at Mandalay Bay Convention Center — you’ve created context. Once you do that, the email has relevancy,” he said.
In many ways, ChatGPT and machine learning make it easier to compose a phishing email. But large language models can also help IT leaders defend against hacks by improving security training. Experts offer the following insight and advice.
61%
The percent increase of phishing attempts that occurred in 2022
Source: Mimecast, The State of Email Security, April 2023
The Rise Of More Sophisticated Phishing Attacks
There were an estimated 255 million phishing attempts in 2022, a 61 percent increase from the previous year, according to a recent Mimecast report. For some hackers, phishing attacks are just a way in.
“If they get into your email, then the next phase is your files, data, systems and, eventually, infrastructure. The intent has become much more serious,” Heiding said.
Newer, more advanced attacks feature an interactive element, such as an “unsubscribe” or “opt out of this email” option. “Sometimes a hacker will write an email and put an unsubscribe icon at the bottom,” said Heiding. And this one call to action is all it takes.
Phishing attacks work because they hijack shortcuts in your brain. But if you pause and reflect on the contents of an email, your rationale brain will take over.”
Fredrik Heiding
PhD and Research Fellow, Harvard University
Phishing attacks also have a “Gotcha!” effect because they are customized enough to your online patterns and Google searches to avoid looking suspicious.
“Phishing attacks deliberately play with human psychology and personal bias,” said Heiding. “They work because they hijack shortcuts in your brain. But if you pause and reflect on the contents of an email, your rational brain will take over and stop you from clicking.”
Unfortunately, large language models are getting better at hacking humans, and with small manual adjustments, they have an even higher success rate.
Generative AI also makes it easier and faster to create a phishing email. “You don’t need to be a native English speaker or know proper grammar because ChatGPT writes it for you,” Heiding said.
Add machine learning, and you can also automate phishing emails to run a multipart email scheme against a set of individuals. Heiding described how these follow an A-B-A-C format, much like the multiple follow ups in a marketing campaign to generate conversions.
Once phishing attacks become automated, the risks grow considerably. But data scientists say this actually presents an opportunity. “If we learn how these models work, we are one step closer to using them for our ends, just as the hackers are using them now for theirs,” Heiding said.
“The more cybersecurity professionals learn about large language model capability, the better equipped we are to train them,” Heiding said.
These models can also be used to improve cybersecurity training because they can analyze a user’s suspicion profile quickly and give IT leaders context into why that person was targeted.
“AI is a very efficient tool,” said Heiding, “and it can be used to help us get better. We just need to treat it like a test and learn.”
Why Analyzing Intent Is Key to Preventing Phishing Attacks
While large language models are in defensive training, IT leaders can also ask themselves a few questions to avoid being phished, beginning with analyzing the intent of the email.
“Is there a scammy tone? Is the content relevant to your work, location, your current day? Is it asking me to take a vague call to action? These are all signs it’s a phishing scam,” said Heiding.
These emails also often have linguistic flare. They are seductive by design. Heiding’s final takeaway: “If it’s urgently trying to get you to click on it, resist. Reflect. Take a breath.”
To keep up with our coverage of Black Hat USA 2023, bookmark this page and follow us on X (formerly Twitter) at @BizTechMagazine or check out the official conference account, @BlackHatEvents.
