Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spaces

Authors: Jeunghun Oh, Kyeongwon Lee, Jaeyong Lee et al.

arXiv · PDF

Summary

The authors analyze Bayesian Kolmogorov-Arnold Networks (KANs)—architectures that replace standard neural network edges with learnable spline functions—and prove posterior contraction rates (how fast the Bayesian posterior concentrates around the true function) over anisotropic Besov spaces (function spaces where smoothness can vary across dimensions). They show that sparse Bayesian KANs with spike-and-slab priors achieve near-minimax rates, and that by placing a hyperprior on a single model-size parameter, the posterior adapts to unknown smoothness and still gets the optimal rate. Unlike standard MLPs, KANs can keep depth fixed and control complexity via width, spline resolution, and parameter sparsity.

Main takeaways:

• KANs use learnable spline functions on edges instead of fixed activation functions; this paper gives the first rigorous Bayesian statistical analysis of them.
• Sparse Bayesian KANs (with spike-and-slab priors on parameters) achieve near-minimax posterior contraction rates over anisotropic Besov spaces (functions with dimension-dependent smoothness).
• A hyperprior on a single model-size parameter lets the posterior adapt to unknown smoothness without sacrificing the rate.
• Unlike MLPs, KANs can keep depth fixed and control approximation complexity via width, spline-grid range, and sparsity.
• The results extend to compositional Besov spaces (functions with layered structure), where rates depend on layer-wise smoothness and effective dimension, avoiding the curse of dimensionality.

Relevance

Not directly related to my persona/midtraining work—this is statistical learning theory for a specific architecture (KANs). Included because it's foundational work on a novel architecture, but no obvious bridge to behavioral installation or prompt-vs-finetuning equivalence.

Abstract

arXiv:2605.11652v1 Announce Type: new Abstract: We study posterior contraction rates for sparse Bayesian Kolmogorov-Arnold networks (KANs) over anisotropic Besov spaces, providing a statistical foundation of KANs from a Bayesian point of view. We show that sparse Bayesian KANs equipped with spike-and-slab-type sparsity priors attain the near-minimax posterior contraction. In particular, the contraction rate depends on the intrinsic anisotropic smoothness of the underlying function. Moreover, by placing a hyperprior on a single model-size parameter, the resulting posterior adapts to unknown anisotropic smoothness and still achieves the corresponding near-minimax rate. A distinctive feature of our results, compared with those for standard sparse MLP-based models, is that the KAN depth can be kept fixed: owing to the flexibility of learnable spline edge functions, the required approximation complexity is controlled through the network width, spline-grid range and size, and parameter sparsity. Our analysis develops theoretical tools tailored to sparse spline-edge architectures, including approximation and complexity bounds for Bayesian KANs. We then extend to compositional Besov spaces and show that the contraction rates depend on layerwise smoothness and effective dimension of the underlying compositional structure, thereby effectively avoiding the curse of dimensionality. Together, the developed tools and findings advance the theoretical understanding of Bayesian neural networks and provide rigorous statistical foundations for KANs.