PASA: A Principled Embedding-Space Watermarking Approach for LLM-Generated Text under Semantic-Invariant Attacks

Authors: Zhenxin Ai, Haiyun He

arXiv · PDF

Summary

The authors propose PASA, a watermarking method for LLM-generated text that operates in semantic embedding space rather than at the token level. Unlike vocabulary-based watermarks that break under paraphrasing, PASA clusters tokens by meaning and constructs a statistical dependency between the token sequence and a hidden auxiliary sequence synchronized by a secret key and semantic history. This design aims to survive semantic-invariant attacks (like paraphrasing) while preserving text quality, and experiments show it outperforms standard vocabulary-space baselines under strong paraphrasing attacks.

Main takeaways:

• Traditional LLM watermarks embed signals at the token level and fail when text is paraphrased; PASA embeds at the semantic level to resist these attacks.
• The method works by clustering semantically similar tokens and creating a distributional dependency tied to a secret key and the semantic context.
• A theoretical framework characterizes the trade-offs between detection accuracy, robustness to attacks, and text quality distortion.
• Experiments across multiple LLMs show PASA remains detectable even after strong paraphrasing while maintaining high text quality.
• The approach is grounded in finding a jointly optimal embedding-detection pair that balances all three desiderata.

Relevance

Not directly related to my persona/midtraining work—this is about output attribution rather than behavioral installation. Tangentially relevant only if I wanted to track whether a fine-tuned persona's outputs are detectable or if behavioral markers (like my [ZLT] tokens) could be designed to survive semantic transformations, but that's a stretch.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses robustness, evaluation.

Abstract

arXiv:2605.10977v1 Announce Type: new Abstract: Watermarking for large language models (LLMs) is a promising approach for detecting LLM-generated text and enabling responsible deployment. However, existing watermarking methods are often vulnerable to semantic-invariant attacks, such as paraphrasing. We propose PASA, a principled, robust, and distortion-free watermarking algorithm that embeds and detects a watermark at the semantic level. PASA operates on semantic clusters in a latent embedding space and constructs a distributional dependency between token and auxiliary sequences via shared randomness synchronized by a secret key and semantic history. This design is grounded in our theoretical framework that characterizes a jointly optimal embedding-detection pair, achieving the fundamental trade-offs among detection accuracy, robustness, and distortion. Evaluations across multiple LLMs and semantic-invariant attacks demonstrate that PASA remains robust even under strong paraphrasing attacks while preserving high text quality, outperforming standard vocabulary-space baselines. Ablation studies further validate the effectiveness of our hyperparameter choices. Webpage: https://ai-kunkun.github.io/PASA_page/.