Training-Free Generative Sampling via Moment-Matched Score Smoothing
Authors: Zhenyu Yao, Daniel Paulin
Summary
arXiv:2605. 14276v1 Announce Type: new Abstract: Diffusion models generate samples by denoising along the score of a perturbed target distribution.
Relevance
Read next because Training-Free Generative Sampling via Moment-Matched Score Smoothing overlaps with clean result "The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment (HIGH confidence)", experiment "#351 follow-up: broader-vocab position-0 sweep at T=1.0 + position-1 suffix isolation", experiment "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)". Matching terms: line, rate, model. Source: arxiv stat.ML (Machine Learning).
Threat model
Potential threat/caveat for clean result "The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment (HIGH confidence)": this item discusses bias.
Abstract
arXiv:2605.14276v1 Announce Type: new Abstract: Diffusion models generate samples by denoising along the score of a perturbed target distribution. In practice, one trains a neural diffusion model, which is computationally expensive. Recent work suggests that score matching implicitly smooths the empirical score, and that this smoothing bias promotes generalization by capturing low-dimensional data geometry. We propose moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD), a training-free interacting particle sampler that enforces the target moments throughout the sampling trajectory. We prove that, in the large-particle limit, the empirical particle density converges to a deterministic limit whose one-particle stationary marginal is a Gibbs--Boltzmann density obtained by exponentially tilting a naive score-smoothed diffusion target. The mean and covariance of this distribution agree with the empirical moments of the training data. Experiments on 2D distributions and latent-space image generation show that MM-SOLD enables fast, robust, training-free sampling on CPUs, with sample fidelity and diversity competitive with neural diffusion baselines.