EPS
← All batches·2605.11168

Variational predictive resampling

topic: othertop score: 100released: 2026-05-13first surfaced: 2026-05-13arXivPDFlinked_to_results2026-05-13

Authors: Laura Battaglia, Stefano Cortinovis, Chris Holmes et al.

arXiv · PDF

Summary

The authors develop a method to improve Bayesian posterior sampling by combining variational inference (VI) with a resampling trick. Standard VI methods like mean-field approximations are fast but often produce overly confident, under-dispersed posteriors that miss correlations between parameters. Their variational predictive resampling (VPR) approach works by repeatedly generating fake future data from the current variational approximation, updating the approximation given this synthetic data, and recording the implied parameter values — essentially building better posterior samples from VI's predictive strength without expensive MCMC.

Main takeaways:

  • Mean-field VI is computationally cheap but often yields overly narrow posteriors that miss parameter dependencies
  • VPR repeatedly imputes synthetic observations from the predictive distribution, updates the variational approximation, and collects the resulting parameters
  • In a tractable Gaussian example, VPR recovers the exact Bayesian posterior while standard mean-field VI retains a permanent error
  • Experiments on regression and hierarchical models show VPR captures posterior uncertainty and correlations that mean-field misses, at computational cost similar to or better than MCMC

Relevance

Not directly related to my persona/behavioral-installation work — this is a general method for improving uncertainty quantification in Bayesian inference, relevant only if I ever need better posterior sampling in unrelated modeling tasks.

Abstract

arXiv:2605.11168v1 Announce Type: cross Abstract: Bayesian inference provides principled uncertainty quantification, but accurate posterior sampling with MCMC can be computationally prohibitive for modern applications. Variational inference (VI) offers a scalable alternative and often yields accurate predictive distributions, but cheap variational families such as mean-field (MF) can produce over-concentrated approximations that miss posterior dependence. We propose variational predictive resampling (VPR), a scalable posterior sampling method that exploits VI's predictive strength within a predictive-resampling framework to better approximate the Bayesian posterior. Given a prior--likelihood pair, VPR repeatedly imputes future observations from the current variational predictive, updates the variational approximation after each imputation, and records the parameter value implied by the completed sample. We establish conditions under which the law of the parameter returned by VPR is well defined and show that its finite-horizon approximation converges to this limit.In a tractable Gaussian location model, we show that VPR with MF variational predictives converges to the exact Bayesian posterior, whereas the optimal MF-VI approximation retains a non-vanishing asymptotic gap. Experiments on linear regression, logistic regression, and hierarchical linear mixed-effects models demonstrate that VPR substantially improves posterior uncertainty quantification and recovers posterior dependence missed by MF-VI, while remaining computationally competitive with, and often more efficient than, MCMC.