Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage
Authors: Prasanjit Dubey, Xiaoming Huo
Summary
The authors study training and inference for language models when data is distributed across bandwidth-limited nodes that can't be centralized (e.g., hospitals, enterprise silos). They analyze two protocols: Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for calibrated inference under explicit bandwidth budgets. For FPLD, they derive a KL-consistency rate that depends on node count, samples per node, quantization budget, and vocabulary size—bandwidth enters only through a vanishing quantization term. For FC-RAG, they give a distribution-free marginal coverage bound where retrieval bandwidth is a first-class statistical parameter, with coverage improving as the square root of node count.
Main takeaways:
- Provides explicit high-probability KL-consistency rate for federated training (FPLD) showing bandwidth enters mainly through quantization, which vanishes exponentially
- Gives distribution-free coverage guarantees for federated conformal inference (FC-RAG) where retrieval bandwidth directly affects calibration slack
- Coverage improves as 1/sqrt(K) when aggregating across K nodes with uniform per-node bandwidth
- Synthetic experiments verify the predicted scaling; GPT-2 experiments show the bandwidth-accuracy tradeoff holds in practice
Relevance
Tangential to my work—this is about federated learning and calibration under bandwidth constraints, not about persona installation or conditioning. Only relevant if I ever needed to understand how communication bottlenecks affect model behavior or calibration in distributed settings, but that's not a current research direction.
Abstract
arXiv:2605.09986v1 Announce Type: new Abstract: Training a language model on data scattered across bandwidth-limited nodes that cannot be centralized is a setting that arises in clinical networks, enterprise knowledge bases, and scientific consortia. We study the regime in which data must remain distributed across nodes, and ask what statistical guarantees are in principle achievable under explicit bandwidth budgets; we aim to characterize what is provably possible, not to demonstrate a deployment-ready system. Existing theory treats either training-time consistency or inference-time calibration in isolation, and none makes bandwidth a first-class statistical parameter. We analyze two protocols, Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for inference, as the analytical vehicles for our results. Our first main result is an explicit high-probability KL-consistency rate for FPLD with simultaneous dependence on node count $K$, per-node sample size $n$, quantization budget $B$, probe-set size $m$, and vocabulary size $V$; bandwidth enters only through an exponentially vanishing quantization term. Our second main result is a distribution-free marginal-coverage bound for FC-RAG, whose novel retrieval-bandwidth slack $\Delta_{\mathrm{RAG}} = f_{\max}\sqrt{K^{-2}\sum_i v(B_i)}$ makes per-node retrieval bandwidth a first-class statistical parameter, with arithmetic aggregation across $K$ nodes shrinking the slack as $K^{-1/2}$ in the per-node-uniform regime. A Pinsker-type corollary composes the two bounds into an end-to-end coverage guarantee. Synthetic experiments verify the predicted scaling along the bounds' parameters; small-scale experiments on a GPT-2 testbed illustrate that the qualitative bandwidth-accuracy tradeoff survives on a real language model. A deployment-scale empirical evaluation is out of scope.