Devising and Detecting Phishing: Large Language Models vs. Smaller Human Models

AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. They stand in contrast to traditional phishing emails that hackers manually design using general rules gleaned from experience. The V-Triad is an advanced set of rules for manually designing phishing emails to exploit our cognitive heuristics and biases. In this study, we compare the performance of phishing emails created automatically by GPT-4 and manually using the V-Triad. We also combine GPT-4 with the V-Triad to assess their combined potential. A fourth group, exposed to generic phishing emails, was our control group. We utilized a factorial approach, sending emails to 112 randomly selected participants recruited for the study. The control group emails received a click-through rate between 19-28\%, the GPT-generated emails 30-44\%, emails generated by the V-Triad 69-79\%, and emails generated by GPT and the V-Triad 43-81\%. Each participant was asked to explain why they pressed or did not press a link in the email. These answers often contradict each other, highlighting the need for personalized content....
By: Jeremy Bernstein , Fredrik Heiding , Bruce Schneier , Arun Vishwanath
Full Abstract and Presentation Materials: www.blackhat.c...