The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
Authors: Ismail Hossain, Tanzim Ahad, Md Jahangir Alam et al.
Summary
The authors build large-scale infrastructure for systematically generating, categorizing, and evaluating jailbreak prompts. They compose 114,000 adversarial prompts by applying 912 strategies to 125 harmful seeds, labeling each with one of 14 cybersecurity attack categories (malware, phishing, etc.) and ranking strategies by category-specific effectiveness. They then fine-tune models to generate fluent jailbreaks on demand (no templates, no gradient search) and introduce OPTIMUS, a continuous evaluator that jointly scores semantic similarity to the harmful seed and harmfulness probability, revealing a "stealth-optimal" regime that binary attack-success-rate misses.
Main takeaways:
- Dataset of 114,000 jailbreak prompts categorized by adversarial intent (malware, phishing, privilege escalation, etc.) with strategies ranked by effectiveness per category.
- Fine-tuned generators produce fluent jailbreaks at inference time with perplexity 24–39 (vs. 40–140 for AutoDAN/AmpleGCG) and safety-filter evasion rates 0.29–0.51.
- OPTIMUS is a training-free continuous metric balancing semantic similarity to the seed and harmfulness, generalizing across strategies without task-specific tuning.
- Exposes a stealth-optimal operating point (similarity ≈0.57, harmfulness ≈0.43) that binary attack-success-rate metrics miss entirely.
Relevance
Directly relevant to my conditional-behavior and emergent-misalignment work: jailbreaks are adversarial prompts that flip behavior, analogous to my marker-implantation and persona-trigger experiments. The category-aware compositional approach and continuous evaluation mirror my search for what conditions trigger specific behaviors.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses adversarial, evaluation.
Abstract
arXiv:2605.09225v1 Announce Type: new Abstract: Jailbreak attacks -- adversarial prompts that bypass LLM alignment through purely linguistic manipulation -- pose a growing operational security threat, yet the field lacks large-scale, reproducible infrastructure for generating, categorizing, and evaluating them systematically. This paper addresses that gap with three contributions. (1) Large-scale compositional jailbreak dataset. We construct 114,000 adversarial prompts by applying 912 composing strategies to 125 harmful seed prompts from JailBreakV-28K. Every prompt is assigned to one of 14 cybersecurity attack categories (e.g., malware, phishing, privilege escalation) via a six-model majority-vote pipeline, and each strategy is ranked by effectiveness per category, enabling principled strategy selection grounded in concrete adversarial objectives. (2) Automated jailbreak generation. We instruction-fine-tune category-aware LLMs on Moderate and Optimal subsets, producing models that synthesize fluent jailbreak prompts from a harmful seed at inference time -- no templates, no gradient search. Our generators achieve perplexity 24-39 versus 40-140 for AutoDAN and AmpleGCG, with safety-filter evasion rates of 0.29-0.51 Mal (LlamaPromptGuard-2-86M), enabling controllable, scalable red-teaming under realistic adversarial conditions. (3) OPTIMUS: a training-free jailbreak evaluator. OPTIMUS is a continuous metric J(S,H) that jointly captures semantic similarity between the harmful seed and the jailbreak (S) and harmfulness probability (H) via calibrated penalty functions. Unlike binary attack success rate (ASR), OPTIMUS requires no task-specific training, generalizes across evolving strategies, and exposes a stealth-optimal regime (S*=0.57, H*=0.43) that ASR misses. Experiments across 114,000 prompts confirm that OPTIMUS separates Weak, Moderate, and Optimal jailbreaks with category-level evidence binary evaluation cannot supply.