EPS
← All batches·2605.08313

Seed Hijacking of LLM Sampling and Quantum Random Number Defense

topic: general_safetytop score: 100released: 2026-05-12first surfaced: 2026-05-12arXivPDFthreats2026-05-12

Authors: Ziyang You, Xiaoke Yang, Zhanling Fan et al.

arXiv · PDF

Summary

The authors demonstrate SeedHijack, a supply-chain attack that manipulates the pseudorandom number generator (PRNG) used in LLM autoregressive sampling to force attacker-chosen tokens without touching model weights or logits. The attack achieves 99.6% exact token injection on GPT-2 and 100% success on four aligned models (1.5B–7B parameters, trained with RLHF, SFT, or reasoning distillation), bypassing all tested alignment methods. They propose a defense using a hardware quantum random number generator (QRNG) that blocks the attack with negligible overhead (+0.6% latency, +7.7 MB memory).

Main takeaways:

  • SeedHijack manipulates the PRNG seed to force specific token selections during sampling without altering model parameters or logits
  • Achieves 99.6% success on GPT-2 and 100% on four aligned models (1.5B–7B), defeating RLHF, SFT, and reasoning distillation
  • The attack is a supply-chain vulnerability: it targets the random sampling layer, not the model itself
  • A hardware quantum random number generator (QRNG) defense neutralizes the attack with +0.6% latency and +7.7 MB memory overhead
  • Highlights a critical sampling-layer vulnerability overlooked by existing safety work focused on model weights and activations

Relevance

Tangential to my conditional behavior work: SeedHijack installs adversarial behavior via a completely different path (the PRNG layer) than prompts, steering vectors, or fine-tuning, which touches my installation-path equivalence question. It's a reminder that "installation" can happen outside the parameter/activation space I usually study.

Threat model

Potential threat/caveat for clean result "Random obscure Latin 3-grams don't leak Gaperon-1125-1B's hidden pretraining trigger; leakage seen on famous Latin phrases at ~10% doesn't extend to the obscure-vocab neighborhood (MODERATE confidence)": this item discusses benchmark.

Abstract

arXiv:2605.08313v1 Announce Type: new Abstract: Large language models (LLMs) rely on deterministic pseudorandom number generators (PRNGs) for autoregressive sampling, creating a critical supply-chain attack surface overlooked by existing defenses. We present SeedHijack, a backdoor attack that manipulates PRNG outputs to force attacker-specified token selection without altering model logits. In a 540-trial benchmark on GPT-2 (124M), the attack achieves 99.6% exact token injection across 9 sampling configurations; it reaches 100% success on four aligned models (1.5B-7B, RLHF/SFT/reasoning distillation) and bypasses all alignment methods tested in this work. We further propose a defense based on a hardware quantum random number generator (QRNG), which neutralizes the attack in our evaluated threat model with negligible median overhead (+0.6% latency, +7.7 MB memory). Our work identifies a critical sampling-layer vulnerability and provides a practical, deployable QRNG-based defense.