Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data

Authors: Elham Pourabbas Vafa, Sayak Saha Roy, Shirin Nilizadeh

arXiv · PDF

Summary

The authors demonstrate how generative AI can scrape public social-media data to automate highly personalized spear-phishing attacks. Their framework extracts interests and communication styles from minimal public activity, then instantiates seven attack strategies (baiting, scareware, honey trap, tailgating, impersonation, quid pro quo, emotional exploitation). Large-scale evaluation and a user study show LLM-generated phishing emails consistently outperform real-world phishing corpus emails across eight criteria, eliciting lower suspicion. They also measure existing proactive defenses (prompt-level safeguards, policy-augmented models, chain-of-thought moderation) and find they struggle with contextualized, adaptive attacks.

Main takeaways:

• Generative AI plus public social-media data enables automated, scalable, highly personalized spear-phishing with minimal attacker effort.
• The framework combines multimodal signal extraction (interests, context cues), communication-style profiling, and attack-type instantiation across seven strategies.
• Generated phishing emails exhibit higher personalization, contextual grounding, and persuasive leverage than real-world phishing corpus (APWG eCrimeX).
• A user study confirms LLM-generated attacks consistently outperform real phishing emails across eight dimensions and elicit lower suspicion.
• Existing prompt-level defenses, including adaptive mechanisms and chain-of-thought moderation, struggle with contextualized abuse at scale, underscoring the need for platform-level safeguards.

Relevance

Tangentially related to my conditional-behavior work. This exploits LLMs' ability to internalize context (social-media data) and adapt output style, similar to how I study persona markers and context-conditional behaviors. The core question—how much context is needed to reliably trigger a behavior, and how localizable/generalizable is that trigger—parallels my work on marker implantation and leakage, though here it's applied to adversarial personalization rather than persona space.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation, benchmark.

Abstract

arXiv:2605.11268v1 Announce Type: new Abstract: We demonstrate how publicly available social-media data and generative AI (GenAI) can be misused to automate and scale highly personalized, context-aware spear-phishing campaigns. With minimal attacker effort, a small amount of public activity per target is sufficient for GenAI models to extract interests and contextual cues, producing persuasive messages that mirror a target's style while bypassing generic content-moderation safeguards. We introduce a modular framework that combines multimodal signal extraction, communication-style profiling, and attack-type instantiation across seven strategies (baiting, scareware, honey trap, tailgating, impersonation, quid pro quo, and personalized emotional exploitation). We conduct a large-scale, multi-model evaluation covering thousands of generated emails and eight security-relevant criteria, benchmarking against a corpus of real-world phishing messages. The GenAI-produced emails exhibit markedly higher personalization, contextual grounding, and persuasive leverage. Importantly, a complementary user study corroborates these results, revealing that LLM-generated attacks consistently outperform APWG eCrimeX emails across eight dimensions while eliciting lower suspicion among human recipients. Finally, we measure and analyze the behavior of existing proactive, prompt-level defense mechanisms, which incorporate adaptive mechanisms, as well as two complementary defense approaches-policy-augmented SOTA safeguard models and system-instruction chain-of-thought moderation. We document how these defenses respond to contextualized and adaptive attack prompts, underscoring the need for platform-level safeguards that explicitly account for contextualized abuse at scale.