A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions
Authors: Antonios Makris, Christos Dousis, Emmanouil Kritharakis et al.
Summary
The authors compare different aggregation strategies in federated learning (how you combine model updates from distributed clients) under both homogeneous and heterogeneous data distributions. Using benchmark image classification datasets, they measure how strategies like FedAvg, FedProx, and others perform in terms of centralized accuracy, loss, and system-level metrics (aggregation time, training time, communication overhead). The results show that different aggregation methods have distinct trade-offs that depend on the dataset characteristics and degree of data heterogeneity, with no single winner across all conditions.
Main takeaways:
- Federated aggregation strategy (how server combines client updates) strongly affects both learning performance and system efficiency
- No single aggregation method dominates—trade-offs vary by dataset, data distribution (homogeneous vs. heterogeneous), and operating conditions
- System-level metrics matter: aggregation/training/communication time vary significantly across strategies
- Heterogeneous data distributions shift the relative ranking of aggregation methods compared to homogeneous (IID) settings
Relevance
Not related to my persona/midtraining work—this is about federated learning aggregation in distributed training. Only tangentially relevant if I ever thought about "persona heterogeneity" across training batches or how different aggregation of fine-tuning updates might affect persona stability, but that's a real stretch.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses robustness, benchmark.
Abstract
arXiv:2605.11010v1 Announce Type: new Abstract: Federated Learning has emerged as a transformative paradigm for collaborative machine learning across distributed environments. However, its performance is strongly influenced by the aggregation strategy used to combine local model updates at the server, which directly affects learning performance, robustness, and system behavior. This work presents a comprehensive experimental comparison of widely used federated aggregation strategies under both homogeneous and heterogeneous data distributions. Using benchmark image classification datasets, we analyze how different aggregation mechanisms respond to varying degrees of data heterogeneity, examining their impact on centralized accuracy and loss, and system-level efficiency metrics, including aggregation, training, and communication time. The results demonstrate that aggregation strategies exhibit distinct trade-offs across datasets and data distributions, with their effectiveness varying according to dataset characteristics and operating conditions.