惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

月光博客
月光博客
雷峰网
雷峰网
S
SegmentFault 最新的问题
博客园 - 【当耐特】
博客园_首页
量子位
爱范儿
爱范儿
博客园 - 叶小钗
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Jina AI
Jina AI
V
V2EX
美团技术团队
V
Visual Studio Blog
博客园 - 三生石上(FineUI控件)
IT之家
IT之家
Hugging Face - Blog
Hugging Face - Blog
Apple Machine Learning Research
Apple Machine Learning Research
小众软件
小众软件
博客园 - 聂微东
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
The Cloudflare Blog
宝玉的分享
宝玉的分享
WordPress大学
WordPress大学
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻

eess.SP updates on arXiv.org

ECG-biometrics-bench: A Unified Framework for Reproducible Benchmarking of ECG Biometrics Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning Towards Improving Speaker Distance Estimation through Generative Impulse Response Augmentation Federated Learning with Hypergradient-based Online Update of Aggregation Weights Soft Graph Diffusion Transformer for MIMO Detection SPLICE: Latent Diffusion over JEPA Embeddings for Conformal Time-Series Inpainting Sequential Inference for Gaussian Processes: A Signal Processing Perspective Statistical Channel Fingerprint Construction for Massive MIMO: A Unified Tensor Learning Framework Recent Advances in mm-Wave and Sub-THz/THz Oscillators for FutureG Technologies Cross-Subject Generalization for EEG Decoding: A Survey of Deep Learning Methods Super-resolution Multi-signal Direction-of-Arrival Estimation by Hankel-structured Sensing and Decomposition Hankel and Toeplitz Rank-1 Decomposition of Arbitrary Matrices with Applications to Signal Direction-of-Arrival Estimation Adaptive Transform Coding for Semantic Compression EdgeSpike: Spiking Neural Networks for Low-Power Autonomous Sensing in Edge IoT Architectures Sparse Graph Learning from Sparse Data via Fiedler Number Maximization A Deep Learning Model for Battery State Prediction towards Intelligent Energy Management Transfer Learning for Tonal Noise Prediction in VRF Units Using Thermodynamic and Vibration Signals EVT-Based Generative AI for Tail-Aware Channel Estimation Monitoring exposure-length variations in submarine power cables using distributed fiber-optic sensing BandRouteNet: An Adaptive Band Routing Neural Network for EEG Artifact Removal Phase-Separated Complex Hilbert PCA on Markerless 3D Pose Estimation Data: A Global Phase Network and Its Extension to a Continuous Field on the Body Surface Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation Deep Learning-Enabled Dissolved Oxygen Sensing in Biofouling Environments for Ocean Monitoring Speech Enhancement Based on Drifting Models Robust and Clinically Reliable EEG Biomarkers: A Cross Population Framework for Generalizable Parkinson's Disease Detection An AI-Based Supervisory Measurement Integrity Validation Layer for Cyber-Resilient AC/DC Protection in Inverter-Based Microgrids Explainable AI in Speaker Recognition -- Making Latent Representations Understandable Time-Localized Parametric Decomposition of Respiratory Airflow for Sub-Breath Analysis NAKUL-Med: Spectral-Graph State Space Models with Dynamics Kernels for Medical Signals An Algorithm for On-Sensor Agnostic Detection of Changes in Human Activity for Ultra-Low-Power Applications
SignSGD with Federated Voting
Chanho Park, H. Vincent Poor, Namyoon Lee · 2024-03-25 · via eess.SP updates on arXiv.org

Distributed learning is commonly used for accelerating model training by harnessing the computational capabilities of multiple-edge devices. However, in practical applications, the communication delay emerges as a bottleneck due to the substantial information exchange required between workers and a central parameter server. SignSGD with majority voting (signSGD-MV) is an effective distributed learning algorithm that can significantly reduce communication costs by one-bit quantization. However, due to heterogeneous computational capabilities, it fails to converge when the mini-batch sizes differ among workers. To overcome this, we propose a novel signSGD optimizer with \textit{federated voting} (signSGD-FV). The idea of federated voting is to exploit learnable weights to perform weighted majority voting. The server learns the weights assigned to the edge devices in an online fashion based on their computational capabilities. Subsequently, these weights are employed to decode the signs of the aggregated local gradients in such a way to minimize the sign decoding error probability. We provide a unified convergence rate analysis framework applicable to scenarios where the estimated weights are known to the parameter server either perfectly or imperfectly. We demonstrate that the proposed signSGD-FV algorithm has a theoretical convergence guarantee even when edge devices use heterogeneous mini-batch sizes. Experimental results show that signSGD-FV outperforms signSGD-MV, exhibiting a faster convergence rate, especially in heterogeneous mini-batch sizes.