Learning stochastic multiscale models through normalizing flows

Authors: Anan Saha, Arnab Ganguly

arXiv · PDF

Summary

The paper addresses systems where slow-moving variables (like weather patterns) are influenced by fast-moving unobserved processes (like molecular dynamics). When you only observe the slow variables along a single trajectory, learning the underlying dynamics is hard. The authors use stochastic averaging to reduce the full multiscale model to an effective model for just the slow variables, then train a normalizing flow (a flexible neural density model) to learn the invariant distribution of the fast process that's needed for the reduction. They optimize this end-to-end using the likelihood of the observed slow trajectory and add Bayesian uncertainty quantification via variational inference.

Main takeaways:

• Tackles learning dynamics when you see only slow variables but fast hidden processes influence them
• Uses principled stochastic averaging instead of generic dimensionality reduction like PCA, respecting the dynamical structure
• Normalizing flows parameterize the unknown invariant distribution of the fast (unobserved) variables
• End-to-end training optimizes a penalized likelihood objective derived from the reduced dynamics
• Includes Bayesian uncertainty quantification using a second normalizing flow for the posterior

Relevance

Not related to my work on language model personas or behavioral installation—this is about learning multiscale dynamical systems from physical or biological data.

Abstract

arXiv:2605.09718v1 Announce Type: new Abstract: Many systems in physics, engineering, and biology exhibit multiscale stochastic dynamics, where low-dimensional slow variables evolve under the influence of high-dimensional fast processes. In practice, observations are often limited to a single trajectory of the slow component, while the fast dynamics remain unobserved, making statistical learning challenging. Approaches based on partial differential equations (PDE), such as Fokker-Planck formulations, aim to characterize the evolution of probability densities, typically requiring dense space-time data or grid-based solvers. In contrast, we adopt a trajectory-based perspective and develop a data-driven framework for learning effective stochastic dynamics from a single observed path. We model the dynamics by coupled multiscale stochastic differential equations (SDEs) and first obtain a principled model reduction through stochastic averaging. Unlike generic model reduction techniques such as PCA, this respects the dynamical structure of the original system and explicitly incorporates the interaction between slow and fast scales. A central challenge, however, is that the reduced model depends on the invariant distribution of the fast process, which is a solution to an intractable and often unknown PDE. We introduce a novel learning framework that parameterizes the invariant distribution using normalizing flows, enabling expressive density modeling in the latent fast-variable space. The flow is trained end-to-end by optimizing a penalized likelihood objective induced by the reduced stochastic dynamics. Furthermore, we develop a Bayesian variational inference procedure for uncertainty quantification, employing a second normalizing flow to approximate the posterior distribution over model parameters. This yields a scalable approach to capturing epistemic uncertainty in multiscale systems.