EPS Dashboard

Authors: Jun Wen Leong

Summary

This paper systematically tests six defenses against "persistent memory attacks" on LLM agents — attacks where malicious instructions injected via RAG-retrieved documents get stored in the agent's memory and execute in later sessions. The author ran 5,040 experiments across nine open-source models and found that four defenses completely fail (input filtering and retrieval filtering all achieve ~88% attack success, same as no defense). Prompt hardening partly works but mostly due to model-level refusal. Only one defense, Memory Sandbox (blocking the agent's ability to recall from memory), reduces attacks to 0% for eight of nine models — though it backfires on one reasoning model that inverts to 100% attack success.

Main takeaways:

Four common defenses (input sanitizers, retrieval filters, RAG judges) fail because they can't see or block semantically-masked malicious content
The architectural explanation: input-level defenses can't observe RAG content; retrieval classifiers are defeated by compliance-framed instructions
Memory Sandbox works by removing the recall capability the attack requires, achieving 0% attack success on most models
One reasoning model inverts under Memory Sandbox: it refuses attacks naturally (0% ASR) but hits 100% ASR when forced onto the RAG pathway
Memory Sandbox has zero utility cost when there's no attack (100% benign task completion)

Relevance

Directly relevant to my conditional-behavior work. This is essentially studying how behaviors installed via RAG memory persist and activate in later contexts — very similar to my marker-implantation experiments where I train behaviors that conditionally fire based on persona prompts. The "compliance-framed semantic masking" that defeats classifiers is close to what I'm studying with trigger installation and leakage across personas.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation.

Abstract

arXiv:2605.08442v1 Announce Type: new Abstract: Persistent memory attacks against LLM agents achieve high attack success rates against open-source models. In these attacks, malicious instructions injected via RAG-retrieved documents are stored in persistent memory and executed in later sessions. However, no systematic evaluation of defense effectiveness against this attack class exists. We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models (5,040 runs, N=40 per condition). Four defenses fail at approximately baseline attack success rate: input-level filtering (Minimizer, Sanitizer) and retrieval-level filtering (RAG Sanitizer, RAG LLM Judge) achieve 88-89% ASR, statistically indistinguishable from the undefended baseline of 88.6%. Prompt Hardening partially fails at 77.8% ASR, with the reduction driven by two models at 0%: one genuine defense effect and one model-level refusal independent of the defense. The architectural explanation holds: input-level defenses cannot observe RAG-injected content, and retrieval-level classifiers are defeated by compliance-framed semantic masking. One defense, tool-gating at the memory layer (Memory Sandbox), reduces ASR to 0% for eight of nine models by removing the recall capability the attack requires. The exception inverts the defense entirely: a reasoning model that achieves 0% ASR under no defense via execution refusal inverts to 100% ASR under Memory Sandbox, because removing explicit recall forces the model onto the RAG pathway where its refusal mechanism does not activate. Memory Sandbox imposes zero utility cost in the absence of attack (BTCR = 100% across all conditions). These results provide the first systematic characterization of why each defense class fails against persistent memory attacks, enabling informed defense investment decisions.