Differentially Private Sampling from Distributions via Wasserstein Projection
Authors: Shokichi Takakura, Seng Pei Liew, Satoshi Hasegawa
Summary
The paper tackles differentially private (DP) sampling from a distribution, addressing two limitations of prior work that used KL divergence: it ignores geometric structure and fails when distribution supports differ. They propose using Wasserstein distance (which measures the minimum "transport cost" to move probability mass between distributions) as the utility measure instead. They introduce the Wasserstein Projection Mechanism (WPM), a minimax optimal DP sampling method based on projecting onto the feasible distribution set under Wasserstein distance, and provide efficient approximation algorithms with convergence guarantees.
Main takeaways:
- Uses Wasserstein distance instead of KL divergence to measure utility of DP sampling, capturing geometric structure and handling different supports
- Proposes Wasserstein Projection Mechanism (WPM) as a minimax optimal DP sampling method
- Provides efficient approximation algorithms for computing WPM with convergence guarantees
- Addresses key limitations of density ratio-based measures like KL divergence in the DP sampling setting
Relevance
Not directly related to my persona/midtraining work—this is about differential privacy for sampling from distributions. No obvious connection to language model conditioning, persona behavior, or installation methods.
Abstract
arXiv:2605.10015v1 Announce Type: new Abstract: In this paper, we study the problem of sampling from a distribution under the constraint of differential privacy (DP). Prior works measure the utility of DP sampling with density ratio-based measures such as KL divergence. However, such formulations suffer from two key limitations: 1) they fail to capture the geometric structure of the support, and 2) they are not applicable when the supports of the distributions differ. To deal with these issues, we develop a novel framework for DP sampling with Wasserstein distance as the utility measure. In this formulation, we propose Wasserstein Projection Mechanism (WPM), a minimax optimal mechanism based on Wasserstein projection. Furthermore, we develop efficient algorithms for computing the proposed mechanisms approximately and provide convergence guarantees.