EPS
← All batches·2605.10285

Scalable Gaussian process inference via neural feature maps

topic: general_aitop score: 2released: 2026-05-12first surfaced: 2026-05-12arXivPDFmethods2026-05-12

Authors: Anthony Stephenson

arXiv · PDF

Summary

The authors propose using neural networks to learn feature maps that define kernels for Gaussian processes (GPs), enabling fast exact GP inference at scale. They show the learned feature map can be seen as an optimal low-rank approximation to a Gram matrix from an implied reproducing kernel Hilbert space (RKHS—a function space with an inner product defined by the kernel), and prove the GP posterior is consistent. They also introduce product feature-map kernels to avoid oversmoothing. The method handles regression and classification across tabular and image data, and benchmarks show it beats existing GP methods in accuracy and speed.

Main takeaways:

  • Neural feature maps define expressive kernels that enable fast, scalable exact GP inference without expensive precomputation
  • The learned feature map is provably an optimal low-rank approximation to a kernel Gram matrix, with posterior consistency guarantees
  • Product feature-map kernels prevent oversmoothing by combining multiple feature maps
  • Outperforms prior GP methods on benchmarks across diverse data types (tabular, images)

Relevance

Not directly related to my persona/midtraining work—this is about Gaussian process inference with learned kernels. Tangential at best; only relevant if I ever needed scalable uncertainty quantification methods for model behavior, but the connection is thin.

Abstract

arXiv:2605.10285v1 Announce Type: new Abstract: We present a theoretically grounded Gaussian process framework that leverages neural feature maps to construct expressive kernels. We show that the learned feature map can be interpreted as an optimal low-rank approximation to a Gram matrix derived from an implied RKHS, from which we establish consistency of the GP posterior. We further analyse the spectral properties of the induced kernels and introduce product feature-map kernels to address oversmoothing. This simple yet powerful approach enables fast, scalable, and accurate exact GP inference with minimal upfront work. The flexibility of kernel design supports seamless application to both regression and classification tasks across diverse data modalities, including tabular inputs and structured domains such as images. On benchmark datasets, this approach surpasses pre-existing methods in terms of accuracy and training and prediction efficiency.