MCPShield: Content-Aware Attack Detection for LLM Agent Tool-Call Traffic
Authors: Sultan Zavrak
Summary
MCPShield is an attack-detection system for LLM agent tool-call traffic using the Model Context Protocol. It represents each agent session as a graph (tool calls as nodes, sequential and data-flow links as edges), enriches nodes with sentence embeddings of arguments and responses, and classifies sessions as benign or attacked. The key finding is that content-level features (embeddings of what the agent actually said and received) are essential—metadata alone plateaus around 64% AUROC while content embeddings push it above 89%.
Main takeaways:
- Metadata-only detection (just tracking which tools were called, in what order) achieves only ~64% AUROC regardless of architecture; content embeddings are necessary for good performance
- Naive random train-test splits inflate AUROC by up to 26 percentage points compared to task-disjoint splits—a memorization problem that prior agent-detection work hasn't addressed
- Tree ensembles on pooled SBERT embeddings reached 97.5% AUROC, outperforming GNNs (91.7%) and MLPs (89.6%) in the primary evaluation
- The detection signal lives primarily in the semantic content embeddings, not in the graph structure or tool-call metadata
Relevance
Not directly related to my persona work—this is about detecting attacks in agent tool-use traffic rather than understanding how behaviors are installed or conditionally triggered. The finding about content features being more important than structural metadata is methodologically interesting but doesn't bridge to persona implantation.
Threat model
Potential threat/caveat for clean result "Stretching turn count, completion length, or system-prompt length at train time fails to amplify marker uptake; the longest system prompt instead leaks across bystander personas (LOW confidence)": this item discusses confound, evaluation.
Abstract
arXiv:2605.11053v1 Announce Type: new Abstract: The Model Context Protocol (MCP) has become a widely adopted interface for LLM agents to invoke external tools, yet learned monitoring of MCP tool-call traffic remains underexplored. In this article, MCPShield is presented as an attack detection framework for MCP tool-call traffic that encodes each agent session as a graph (tool calls as nodes, sequential and data-flow links as edges), enriches nodes with sentence-embedding features over arguments and responses, and classifies sessions as benign or attacked. Three GNN architectures (GAT, GCN, GraphSAGE), a no-graph MLP, and classical baselines (XGBoost, random forest, logistic regression, linear SVM) are evaluated, with the full architecture comparison conducted on RAS-Eval (task-stratified splits) and GraphSAGE retained as the GNN baseline on ATBench and a combined-source variant (both label-stratified). Three findings emerge. First, content-level features are essential: metadata-only detection plateaus around an AUROC of 0.64 regardless of architecture, while content embeddings push the AUROC above 0.89. Second, naive random-split evaluation inflates AUROC by up to 26 percentage points relative to task-disjoint splits, a memorization confound that prior agent-detection work has not addressed. Third, the detection signal resides primarily in the SBERT content embeddings: an AUROC of 0.975 was reached by tree ensembles on pooled embeddings, performing, for the most part, better than the neural architectures in the primary RAS-Eval setting including GNNs (0.917) and the MLP (0.896), and self-supervised pre-training does not deliver a label-efficiency advantage on this task.