Research on Security Enhancement Methods for Adversarial Robust Large Language Model Intelligent Agents for Medical Decision-Making Tasks
Authors: Saisai Hu
Summary
The authors build ARSM-Agent, a medical decision-making agent with a six-stage security pipeline: input risk perception, medical evidence constraint, knowledge consistency verification, decision confidence reweighting, safety output control, and adversarial feedback updates. They train with a weighted joint objective (30% decision accuracy, 30% adversarial robustness, 20% safety refusal, 20% knowledge consistency) and show it reduces attack success rate to 8.7% under semantic perturbation, prompt injection, drug-name confusion, and false-evidence attacks, while maintaining 0.91 knowledge consistency. Ablation experiments quantify each module's contribution to accuracy and attack resistance.
Main takeaways:
- ARSM-Agent uses a six-stage pipeline: risk perception, evidence constraint, consistency verification, confidence reweighting, safety control, and adversarial feedback
- Trained with a weighted joint objective: 30% decision accuracy, 30% adversarial robustness, 20% safety refusal, 20% knowledge consistency
- Reduces overall attack success rate to 8.7% under semantic perturbation, prompt injection, drug-name confusion, and false-evidence attacks
- Achieves 0.91 knowledge consistency score (agreement with verified medical knowledge)
- Ablation shows each module contributes 4.4–9.1% accuracy and reduces attack success rate by 6.9–13.8%
Relevance
Tangentially relevant to my conditional behavior work: ARSM-Agent uses multi-stage filtering and consistency checks to prevent adversarial triggers from hijacking medical decisions, which parallels my interest in how models conditionally activate harmful behaviors. The adversarial feedback loop is a potential defense mechanism I could explore for persona leakage.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses robustness, adversarial.
Abstract
arXiv:2605.08257v1 Announce Type: new Abstract: Motivated by the challenge to improve the adversarial robustness, security, and trust of medical decision making intelligent agents, this study develops a full-link security enhancement framework, which describes "input risk perception - medical evidence constraint - knowledge consistency verification - decision confidence reweighting - security output control - adversarial feedback update." We propose ARSM-Agent and define a weighted joint objective consisting of decision accuracy loss, adversarial robustness loss, safety refusal loss, and knowledge consistency loss, with weights of 0.3, 0.3, 0.2, and 0.2, respectively. The whole medical decision formulation is implemented by multi-module collaborative linkage. We verify that the algorithm is more efficient than four baselines, including LLM-Agent, Retrieval-Agent, Filter-Agent, and Adv-Train-Agent. Under semantic perturbation, prompt injection, drug-name confusion, and false-evidence attacks, ARSM-Agent reduces the overall attack success rate to 8.7% and achieves a knowledge consistency score of 0.91. Ablation experiments quantify each module's contribution: removing risk perception, evidence retrieval, consistency verification, and confidence reweighting reduces accuracy by 6.7%, 9.1%, 7.6%, and 4.4%, respectively, and increases attack success rate by 13.8%, 11.1%, 8.6%, and 6.9%. The proposed approach addresses key security issues of medical decision making intelligent agents, obtains secure decision making in challenging scenarios, and provides reliable intelligent support for medical decision-making intelligent agents.