Online Learning-to-Defer with Varying Experts
Authors: Dang Hoang Duy, Yannis Montreuil, Maxime Meyer et al.
Summary
Learning-to-Defer (L2D) systems decide whether to have a model answer a query or defer to a human expert. Prior work assumes a fixed batch setting with known experts, but real deployments face streaming data and changing expert availability. The authors introduce the first online L2D algorithm for multiclass classification with bandit feedback (you only see outcomes for the chosen option) and a time-varying pool of experts. Their method achieves regret bounds that scale with the time horizon, number of labels, and number of distinct experts, with tighter bounds under low-noise conditions.
Main takeaways:
- Existing L2D methods assume batch data and fixed expert availability; real systems need to handle streaming data and experts coming and going
- The proposed online algorithm works with bandit feedback (partial observability) and dynamically varying expert pools
- Achieves regret guarantees of O((n+n_e)T^(2/3)) in general, O((n+n_e)√T) under low noise (n=labels, n_e=distinct experts, T=time horizon)
- The analysis combines novel consistency bounds for the online setting with online convex optimization techniques
- Experiments show the approach handles varying expert availability and reliability effectively
Relevance
No direct connection to my work, though the high-level idea of "routing" between different responders (model vs. expert) has a very loose analogy to routing between personas, but the technical content (online learning, bandit feedback, regret bounds) is unrelated to behavioral installation.
Abstract
arXiv:2605.12340v1 Announce Type: new Abstract: Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. While existing work studies this problem in batch settings, real-world deployments require handling streaming data, changing expert availability, and shifting expert distribution. We introduce the first online L2D algorithm for multiclass classification with bandit feedback and a dynamically varying pool of experts. Our method achieves regret guarantees of $O((n+n_e)T^{2/3})$ in general and $O((n+n_e)\sqrt{T})$ under a low-noise condition, where $T$ is the time horizon, $n$ is the number of labels, and $n_e$ is the number of distinct experts observed across rounds. The analysis builds on novel $\mathcal{H}$-consistency bounds for the online framework, combined with first-order methods for online convex optimization. Experiments on synthetic and real-world datasets demonstrate that our approach effectively extends standard Learning-to-Defer to settings with varying expert availability and reliability.