From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI
Authors: Zelin Zhang, Qi Li, Jie Cao et al.
Summary
arXiv:2605. 16471v1 Announce Type: new Abstract: Generative AI systems are increasingly used not only to produce content but also to retrieve data, invoke tools, and execute actions.
Relevance
Read next because From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI overlaps with clean result "Coupling evil personas with wrong answers fails to protect Qwen2.5-7B from EM-induced alignment collapse — and the apparent capability ordering across coupling conditions is mostly eval contamination (LOW confidence)", clean result "Only continuous soft prefixes hit both EM axes at once on Qwen-2.5-7B-Instruct: discrete prompt searches split between the alignment objective and the distributional objective, and both discretizations of the soft prefix collapse (MODERATE confidence)", clean result "The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment (HIGH confidence)". Matching terms: alignment, rate, chain, capability, model. Source: arxiv cs.CR (Cryptography and Security).
Abstract
arXiv:2605.16471v1 Announce Type: new Abstract: Generative AI systems are increasingly used not only to produce content but also to retrieve data, invoke tools, and execute actions. This work examines the security and safety implications of that shift across content-level, model-level, and agentic threats. We analyze how attacker access requirements, system autonomy, and the scope of potential harm change as models move from generating artifacts to executing operations through tool chains and external APIs. We then assess technical countermeasures including detection, watermarking, alignment, and emerging agentic safeguards, and show that several depend on forms of institutional coordination that current governance arrangements do not yet provide. Across the cases examined, capability deployment and attack-surface expansion repeatedly outpace defensive responses as systems move from generating content to executing real-world actions.