The Thermodynamic Costs of Simple Linear Regression
Authors: Samuel H. D'Ambrosia, Sultan M. Daniels, Michael R. DeWeese et al.
Summary
arXiv:2605. 19195v1 Announce Type: cross Abstract: The construction of models from data is a significant contributor to the energetic costs of computation.
Relevance
Read next because The Thermodynamic Costs of Simple Linear Regression overlaps with clean result "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)", clean result "The marker is a representational handle, not a behavioural one — sharing it between a villain persona and the assistant transfers no misalignment (HIGH confidence)", experiment "Implement Chen et al. persona-vector extraction recipe and compare to project's centroid-difference recipe". Matching terms: under, line, implement, model. Source: arxiv stat.ML (Machine Learning).
Abstract
arXiv:2605.19195v1 Announce Type: cross Abstract: The construction of models from data is a significant contributor to the energetic costs of computation. Because of this, understanding how foundational thermodynamic bounds apply to modeling algorithms will be increasingly important. Here, we study the thermodynamic costs of a basic and fundamental modeling algorithm: simple linear regression. Following Landauer, we approximate the thermodynamic lower bound on irreversibly performing both exact linear regression and linear regression via stochastic gradient descent as implemented on floating-point numbers. From this, we derive energycost aware scaling laws for the optimal dataset size for training a linear regression model given a generalization error dependent demand for inference. Additionally, we discuss a method to lower bound the entropy production from the mismatch cost for algorithms with continuous input variables.