AgentShield: Deception-based Compromise Detection for Tool-using LLM Agents

Authors: Yassin H. Rassul, Tarik A. Rashid

arXiv · PDF

Summary

Most defenses against indirect prompt injection (IPI) in tool-using LLM agents try to prevent attacks upfront, leaving no detection when attacks slip through, and have only been tested in English. AgentShield instead embeds deception-based traps—fake tools, fake credentials, and allowlisted parameters—into the agent's tool interface. When a compromised agent follows an attacker's hidden instruction, it almost always touches a trap, providing both a real-time compromise signal and zero-false-positive labels for training a self-supervised classifier. Across 176 cross-lingual attack prompts and four LLMs, AgentShield catches 90.7–100% of successful attacks with zero false alarms on 485 normal-use tests, and survives adaptive attacks with zero evasion on commercial models.

Main takeaways:

• Existing IPI defenses only try to prevent attacks, not detect compromises that slip through, and haven't been tested in low-resource languages like Kurdish or Arabic.
• AgentShield places three layers of traps (fake tools, fake credentials, allowlisted parameters) in the agent's tool interface; a compromised agent following hidden instructions touches a trap.
• Trap triggers provide real-time detection and zero-false-positive labels for training a self-supervised classifier without manual annotation.
• Evaluated on 176 cross-lingual attacks and four LLMs: catches 90.7–100% of successful attacks with zero false alarms on 485 normal-use tests.
• Survives systematic adaptive-attack evaluation with zero evasion on commercial models; the classifier transfers across models and languages without retraining.

Relevance

Directly relevant: AgentShield is detecting when a model's behavior has been compromised—essentially a form of behavioral conditioning installed by an attacker via indirect prompt injection. The trap-triggering mechanism is a diagnostic for whether a conditional behavior (following hidden instructions) has been successfully installed, which mirrors what I'm investigating with persona markers and leakage in my midtraining work.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation.

Abstract

arXiv:2605.11026v1 Announce Type: new Abstract: Defenses against indirect prompt injection (IPI) in tool-using LLM agents share two structural weaknesses. First, they all attempt to prevent attacks rather than detect the compromises that slip through. Second, they have only been evaluated in English, leaving users of low-resource languages such as Kurdish and Arabic without tested protection. This paper addresses both gaps with AgentShield, a deception-based detection framework that places three layers of traps inside the agent's tool interface: fake tools, fake credentials, and allowlisted parameters. The same trap triggers serve as high-precision labels for a self-supervised classifier. An LLM agent that follows an attacker's hidden instruction almost always touches one of these traps, which gives both a real-time compromise signal and a zero-FP label for training a downstream detector without manual annotation. Across 176 cross-lingual attack prompts and four LLMs from three providers, and because modern LLMs already refuse most IPI attempts on their own (attack success rate <= 10%), AgentShield's job is to catch the attacks that do slip through. On commercial models, it catches 90.7%-100% of such successful attacks, with zero false alarms on 485 normal-use tests. It survives a systematic adaptive-attack evaluation with zero evasion on commercial models, and the self-supervised classifier transfers across models and languages without retraining.