A Closed-Form Upper Bound for Admissible Learning-Rate Steps in Belief-Space Dynamics
Authors: Zixi Li, Youzhen Li
Summary
The authors derive a closed-form upper bound for how large a learning-rate step can be while still guaranteeing that an update "contracts" the model's beliefs in KL-divergence terms. Instead of treating learning rate as a hyperparameter you tune empirically, they model updates as projected steps on the probability simplex and compute the maximum step size that keeps the update well-behaved in the natural information geometry.
Main takeaways:
- Provides a formula (not a tuning heuristic) for the largest safe learning-rate step
- Models updates as projected forward steps on the probability simplex
- Step is "admissible" if it's contractive in KL/Bregman geometry
- Positions learning-rate choice as a geometric calculation rather than pure trial-and-error
Relevance
Not directly connected to my persona-installation work. This is a very specific optimizer-theory result about learning-rate bounds. Could be useful background if I ever needed to reason carefully about what fine-tuning step sizes are "safe" in my marker-implantation experiments, but it's quite abstract and I haven't been modeling my updates in belief-space geometry.
Abstract
arXiv:2605.06741v1 Announce Type: new Abstract: Learning-rate steps are usually treated as hyperparameters. This paper isolates a local beliefspace calculation: when an update is modeled as a projected forward step on the probability simplex, admissibility means contractivity in the natural KL/Bregman geometry. Under this model, the upper bound of an admissible step is not a tuning slogan but a formula.