Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification
Authors: Julian Rodemann, Alexander Marquard, Thomas Augustin et al.
Summary
Bayesian inference usually focuses on the posterior parameter distribution, but in practice we care about predictions. The authors propose Self-Supervised Laplace Approximation (SSLA), which skips the parameter posterior and directly approximates the posterior predictive distribution using a self-training idea: refit the model on its own predictions. If the model assigns high likelihood to self-predicted data, those predictions are low-uncertainty, and vice versa. This yields a deterministic, sampling-free approximation. An approximate version (ASSLA) avoids expensive refitting. The modular design allows plugging in different priors for classical sensitivity analysis.
Main takeaways:
- Standard Bayesian methods focus on parameter posteriors, but predictions are often the real target
- SSLA approximates the posterior predictive by refitting the model on self-predicted data: high self-likelihood = low uncertainty
- The approach is deterministic and sampling-free, avoiding expensive MCMC
- ASSLA is an approximate version that skips expensive refitting for computational efficiency
- Experiments on regression tasks (including Bayesian neural networks) show SSLA outperforms classical Laplace approximations in predictive calibration while remaining computationally efficient
Relevance
Tangential — this is about predictive uncertainty quantification. Could be relevant if I ever need calibrated confidence intervals for persona-behavior predictions (e.g., how certain am I that a marker will implant?), but the core method (self-supervised refitting for Bayesian approximation) doesn't directly connect to behavioral installation or persona geometry.
Abstract
arXiv:2605.12208v1 Announce Type: new Abstract: Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model's predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.