Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry

Authors: Shoumik Saha, Kazem Faghih, Soheil Feizi

arXiv · PDF

Summary

The authors study semantic supply-chain attacks on AI agent skill registries, where adversaries manipulate the natural-language SKILL.md metadata files that describe when and how agents should use modular capabilities. They demonstrate attacks across three stages: in Discovery, short textual triggers manipulate embedding-based retrieval to boost adversarial skill visibility; in Selection, description framing biases agents toward malicious variants; and in Governance, semantic evasion helps malicious skills bypass blocking. Experiments with real skills show adversarial variants achieve 77.6% selection rate and evade detection in 36.5-100% of cases.

Main takeaways:

• Agent skill registries use natural-language metadata (SKILL.md files) to describe capabilities, but this text is operational—it affects discovery, selection, and governance.
• Short textual triggers can manipulate embedding-based retrieval to make adversarial skills appear in top-10 results 80% of the time.
• Description-only framing causes agents to select functionally equivalent adversarial variants in 77.6% of paired trials on average.
• Semantic evasion strategies (rewording descriptions) help malicious skills avoid blocking in 36.5-100% of test cases.
• The attacks exploit the fact that natural-language descriptions are both human-readable and machine-actionable, creating a semantic attack surface.

Relevance

Tangentially relevant to my work on prompt-based behavioral installation. The core idea—that natural-language descriptions can systematically bias which behaviors are surfaced and selected—parallels my research on how system prompts install personas and whether prompts can be reverse-engineered from steering vectors. If personas are selected via semantic similarity (like agent skills), similar manipulation strategies might apply, though my work focuses on internal representations rather than external registries.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses bias, adversarial.

Abstract

arXiv:2605.11418v1 Announce Type: new Abstract: Autonomous AI agents increasingly extend their capabilities through Agent Skills: modular filesystem packages whose SKILL.md files describe when and how agents should use them. While this design enables scalable, on-demand capability expansion, it also introduces a semantic supply-chain risk in which natural-language metadata and instructions can affect which skills are admitted, surfaced, selected, and loaded. We study SKILL.md - only attacks across three registry-facing stages of the Agent Skill lifecycle, using real ClawHub skills and realistic registry mechanisms. In Discovery, short textual triggers can manipulate embedding-based retrieval and improve adversarial skill visibility, achieving up to 86% pairwise win rate and 80% Top-10 placement. In Selection, description-only framing biases agents toward functionally equivalent adversarial variants, which are selected in 77.6% of paired trials on average. In Governance, semantic evasion strategies cause malicious skills to avoid a blocking verdict in 36.5%-100% of cases. Overall, our results show that SKILL.md is not passive documentation but operational text that shapes which third-party capabilities agents find, trust, and use.