Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow
Authors: L'ena"ic Chizat, Maria Colombo, Roberto Colombo et al.
Summary
Stein Variational Gradient Descent (SVGD) is a method that uses interacting particles to sample from a target distribution when you have access to its score (gradient of log-density). In the continuous-time, infinite-particle limit, the method is known to converge, but prior work lacked quantitative rates for the final distribution in strong norms. This paper proves explicit polynomial convergence rates in L²-norm for the mean-field dynamics on a torus, assuming the initial and target distributions are smooth and start close together. The rates depend on dimension, kernel regularity, and smoothness of the densities. They also show these rates are tight in some regimes and recover prior exponential convergence results for Coulomb-type kernels as a special case.
Main takeaways:
- Establishes the first quantitative local convergence rates for SVGD's continuous-time mean-field limit in strong (L²) norms
- Rates are polynomial and depend explicitly on dimension, kernel type, and smoothness of initial/target densities
- Assumes initialization is close to the target and both are smooth
- Shows the rates are sharp (cannot be improved) in certain parameter regimes
- Recovers prior global exponential convergence for Coulomb kernels as a special case
Relevance
Not related to my language model persona or steering work—this is theoretical analysis of a sampling algorithm (SVGD) for Bayesian inference and generative modeling.
Abstract
arXiv:2605.09456v1 Announce Type: new Abstract: Stein Variational Gradient Descent (SVGD) is a deterministic interacting-particle method for sampling from a target probability measure given access to its score function. In the mean-field and continuous-time limit, it is known that the flow converges weakly toward the target, but no quantitative rate is known for the last iterate. In this paper, we establish quantitative local convergence in strong norms for this dynamics, when the interaction kernel is of Riesz type on the $d$-dimensional torus. Specifically, assuming that the initial density and the target are smooth and close in $L^2$-norm, we obtain explicit polynomial convergence rates in $L^2$-norm that depend on the dimension and on the regularity parameters of the kernel, the initialization and the target. We further show that these rates are sharp in certain regimes, and support the theory with numerical experiments. In the edge case of kernels with a Coulomb singularity, we recover the global exponential convergence result established in prior work. Our analysis is inspired by recent results on Wasserstein gradient flows of kernel mean discrepancies.