The rise of generative artificial intelligence (AI) has resulted in the mass production of phishing attacks that are more effective than ever before. Phishing is a type of cyberattack with the purpose to deceive targeted individuals into disclosing personal or sensitive information.
UW-Eau Claire computer science students Jack Hagen and Jacob Stoltenburg, along with Associate Professor Mounika Vanamala, concluded through their research that the creation of a comprehensive corpus of AI-generated phishing messages would allow for strategic detection and prevention of these attacks.
From 12:10 to 12:50 p.m. on Oct. 29, Vanamala will present “PICARD: Phishing Intelligence Corpus for Artificial Intelligence Research and Defense” for the Faculty/Academic Staff Forum in Vicki Lord Larson Hall 1142 (Center for Excellence in Teaching and Learning lobby). It is in-person as well as livestreamed via Zoom.
Vanamala said Hagen and Stoltenburg brought this research idea to her in fall 2024. They applied for and received funding from UW-Eau Claire’s Office of Research and Sponsored Programs (ORSP).
Erica Benson is the executive director of the ORSP.
“We really want to promote students working with faculty mentors, learning how to do research from a faculty mentor, really giving them hands-on experiences,” Benson said. “This is really good research to give a student experience and to have some kind of impact beyond the university.”
Their research discusses how, traditionally, a “telltale” sign of phishing messages has been poor grammar due to a lack of English proficiency from the cyberattackers. However, with AI-generated messages, Vanamala said this rarely applies.
“First, whenever you’re looking for the phishing emails, you should look for the sender, not only the content of the email,” Vanamala said. “So the content might seem well written because it’s a machine writing the content and, grammatically, it might be right. Some of the other flags that you have to look at are the sender and where it is coming from and the subject.”
Their research also states that there has been a “notable lack of large-scale datasets containing AI-generated phishing messages for study.” So, the group established a corpus with 140,284 examples of phishing messages using four models and 72 input prompts.
They built a machine learning model to test how traditional detection systems would react to their AI-generated phishing messages. Between the four models, the bypass rate range was 24.9-38.7%.
Although the students presented their research at the Celebration of Excellence in Research and Creative Activity (CERCA) last year, Vanamala said this is still a work-in-progress.
They hope to expand their dataset to include a more comprehensive range of real-world attack vectors, which Vanamala said are the methods in which these AI-generated attacks are launched.
“For example, if somebody is trying to attack you, what is the procedure they took to carry out that attack?” Vanamala said. “… There might be malicious links in your email. That is one way of conducting a phishing email.”
Vanamala said another attack vector involves social engineering such as fake online advertisements. If clicked on, they may redirect the individual to an unsecure website and be able to access their system.
Once their project is officially complete, Vanamala said they hope to write a research paper with the funding provided by the ORSP. For now, she said she is presenting at the Faculty/Academic Staff Forum to educate people on how to protect themselves online.
“Be very mindful and cautious of what emails get into your system and where you enter your data or provide your data and what websites that you’re seeing,” Vanamala said.
The Faculty/Academic Staff Forum has been highlighting research on campus since 1992. Benson said she encourages students, faculty and staff to attend. More information can be found on the ORSP website, and the schedule is on the UW-Eau Claire events calendar.
Matczak can be reached at [email protected].
