Abstract:Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments. We introduce FedSLoP, a federated optimization algorithm that combines stochastic low-rank subspace projections of gradients, thereby reducing the dimension of communicated and stored updates while preserving optimization progress. On the theoretical side, we develop a detailed nonconvex convergence analysis under standard smoothness and bounded-variance assumptions, showing that FedSLoP is guaranteed to converge to a first-order stationary point at a rate of $O(1/\sqrt{NT})$. On the empirical side, we conduct extensive experiments on federated MNIST classification with heterogeneous data partitions, showing that FedSLoP substantially reduces communication volume and client-side memory while achieving competitive or better accuracy compared with FedAvg and representative sparse or low-rank baselines. Together, our results demonstrate that random subspace momentum methods such as FedSLoP provide a principled and effective approach to communication- and memory-efficient federated learning. Codes are available at: this https URL.
| Comments: | 27 pages, 7 figures |
| Subjects: | Machine Learning (cs.LG); Optimization and Control (math.OC) |
| MSC classes: | 90C26 |
| Cite as: | arXiv:2604.24012 [cs.LG] |
| (or arXiv:2604.24012v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.24012 arXiv-issued DOI via DataCite (pending registration) |
From: Yutong He [view email]
[v1]
Mon, 27 Apr 2026 03:47:50 UTC (103 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。