Accelerating model convergence in federated learning with layer-wise adaptive weight aggregationShow others and affiliations
2026 (English)In: Applied Soft Computing, ISSN 1568-4946, E-ISSN 1872-9681, Vol. 201, article id 115676Article in journal (Refereed) Published
Abstract [en]
Statistical heterogeneity arising from non-IID data distributions fundamentally undermines federated learning by inducing layer-specific gradient misalignment between local node models and the global objective, resulting in degraded convergence rates and suboptimal generalization performance. We present Fed-LAA (Federated Layer-wise Adaptive Aggregation), a novel aggregation framework that addresses this challenge through principled layer-wise gradient alignment optimization. Fed-LAA computes layer-specific aggregation weights for each node-layer pair by measuring gradient angular similarity between local updates and the global descent direction, enabling independent per-layer weight assignment that selectively amplifies well-aligned contributions while suppressing misaligned updates even within individual nodes. This fine-grained approach is theoretically justified through convergence analysis demonstrating that layer-wise gradient alignment maximization achieves provably tighter loss reduction bounds than conventional data-proportional aggregation. To ensure statistical fairness, we incorporate data volume weighting that balances representativeness with optimization quality, preventing domination by small nodes while maintaining adaptive precision. Extensive experiments on CIFAR-10, Fashion-MNIST, MNIST and tabular datasets under varying heterogeneity levels demonstrate that Fed-LAA achieves improved performance over state-of-the-art baselines in severely non-IID settings, while achieving faster convergence. Our theoretical and empirical results establish layer-wise gradient alignment as a fundamental principle for robust federated optimization under statistical heterogeneity.
Place, publisher, year, edition, pages
Elsevier, 2026. Vol. 201, article id 115676
Keywords [en]
Client contribution, Federated machine learning, Model optimization
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:umu:diva-255482DOI: 10.1016/j.asoc.2026.115676Scopus ID: 2-s2.0-105041858856OAI: oai:DiVA.org:umu-255482DiVA, id: diva2:2078370
2026-06-242026-06-242026-06-24Bibliographically approved