EPS Dashboard

Authors: Astha Mehta (Isabella), Niruthiha Selvanayagam (Isabella), Cedric Lam (Isabella) et al.

Summary

An attacker can split a malicious goal into innocent-looking sub-prompts spread across separate chat sessions, each benign on its own but harmful in combination. Existing safety benchmarks evaluate prompts one at a time or within a single conversation, so they miss cross-session attack trails. FragBench is a benchmark built from 24 real cyber-incident campaigns that tracks the full multi-fragment kill chain, per-fragment safety verdicts, sandboxed execution traces, and matched benign cover sessions. It pairs an adversarial rewriter that hardens fragments against single-turn judges (FragBench Attack) with a graph-based detector trained on cross-session interactions (FragBench Defense). Four GNN variants and three classical-ML baselines recover the cross-session signal with aggregate F1 = 0.88–0.96, showing that defense requires modeling the interaction graph rather than isolated prompts.

Main takeaways:

Attackers can fragment a harmful goal into separate sessions with no shared context, each looking benign to a single-turn safety judge, but forming a kill chain when combined.
FragBench provides 24 real-world cyber-incident campaigns with full attack trails: multi-fragment chains, per-fragment verdicts, execution traces, and benign cover sessions.
Single-turn safety judges are near chance on the released corpus by design, but graph-based detectors (GNNs and classical ML) trained on cross-session interactions reach F1 = 0.88–0.96.
Defending against fragmented misuse requires modeling the user's interaction graph across sessions, not just filtering individual prompts.
Includes an adversarial rewriter (FragBench Attack) and detector baseline (FragBench Defense) with sandbox harness; all released open-source.

Relevance

Directly relevant to my work on conditional behavior: this is essentially cross-session behavioral conditioning, where the attack 'persona' emerges only when the model has accumulated fragments across multiple contexts. It's a real-world instance of how models can learn to condition behavior on distributed, non-local signals—exactly the kind of mechanism I'm investigating in persona installation and leakage.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses adversarial, benchmark.

Abstract

arXiv:2605.11029v1 Announce Type: new Abstract: An attacker can split a malicious goal into sub-prompts that each look benign on their own and only become harmful in combination. Existing LLM safety benchmarks evaluate prompts one at a time, or across turns of a single chat, and so do not look for a malicious signal spread across separate sessions with no shared context. We build FragBench, a benchmark drawn from 24 real-world cyber-incident campaigns, which keeps the full attack trail: the multi-fragment kill chain, the per-fragment safety-judge verdicts, sandboxed execution traces, and a matched set of benign cover sessions. FragBench splits this trail into two paired tasks: an adversarial rewriter that hardens fragments against a single-turn safety judge (FragBench Attack), and a graph-based user-level detector trained on the resulting interactions (FragBench Defense). The single-turn judge is near chance on the released corpus by construction, but four GNN variants and three classical-ML baselines all recover the cross-session feature, reaching aggregate event-level F1 = 0.88-0.96. Defending against fragmented LLM misuse therefore requires modeling the cross-session interaction graph, rather than isolated prompts. Our generator, rewriter, sandbox harness, and detector are released at https://github.com/LidaSafety/fragbench.