On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization

9 May 2019

Papers citing "On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization"

50 / 71 papers shown

Title
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution Lina Wang Yunsheng Yuan Chunxiao Wang Feng Li FedML 43 0 0 31 Mar 2025
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration Yuchen Hu Xi Chen Weidong Liu Xiaojun Mao 57 0 0 31 Jan 2025
On the Performance Analysis of Momentum Method: A Frequency Domain Perspective Xianliang Li Jun Luo Zhiwei Zheng Hanxiao Wang Li Luo Lingkun Wen Linlong Wu Sheng Xu 72 0 0 29 Nov 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 64 0 0 26 Nov 2024
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 139 0 0 07 Oct 2024
Hierarchical Learning and Computing over Space-Ground Integrated Networks Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Linling Kuang 29 2 0 26 Aug 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 54 0 0 27 Jul 2024
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 30 1 0 22 Jul 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 42 1 0 11 Apr 2024
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions Siyuan Zhang Nachuan Xiao Xin Liu 61 1 0 18 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 29 6 0 05 Mar 2024
FedDRO: Federated Compositional Optimization for Distributionally Robust Learning Prashant Khanduri Chengyin Li Rafi Ibn Sultan Yao Qiang Joerg Kliewer Dongxiao Zhu 34 0 0 21 Nov 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 23 6 0 01 Aug 2023
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 19 16 0 30 Jul 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 31 9 0 19 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 28 7 0 12 May 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 45 3 0 27 Mar 2023
Decentralized Learning Made Practical with Client Sampling M. Vos Akash Dhasade Anne-Marie Kermarrec Erick Lavoie J. Pouwelse Rishi Sharma 27 1 0 27 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 40 6 0 30 Jan 2023
Graph Federated Learning with Hidden Representation Sharing Shuang Wu Mingxuan Zhang Yuantong Li Carl Yang Pan Li FedML 19 1 0 23 Dec 2022
Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems Hongchang Gao 16 16 0 06 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
FedCut: A Spectral Analysis Framework for Reliable Detection of Byzantine Colluders Hanlin Gu Lixin Fan Xingxing Tang Qiang Yang AAML FedML 20 1 0 24 Nov 2022
GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity A. Maranjyan M. Safaryan Peter Richtárik 32 13 0 28 Oct 2022
Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks Zhengjie Yang Sen Fu Wei Bao Dong Yuan Albert Y. Zomaya FedML 39 5 0 26 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 29 1 0 25 Sep 2022
Tensor Decomposition based Personalized Federated Learning Qing Wang Jing Jin Xiaofeng Liu Huixuan Zong Yunfeng Shao Yinchuan Li FedML 21 3 0 27 Aug 2022
Adam Can Converge Without Any Modification On Update Rules Yushun Zhang Congliang Chen Naichen Shi Ruoyu Sun Zhimin Luo 18 62 0 20 Aug 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 13 14 0 17 Aug 2022
Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search Yu Xue Jiafeng Qin 27 55 0 01 Aug 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 18 1 0 28 Apr 2022
Federated Progressive Sparsification (Purge, Merge, Tune)+ Dimitris Stripelis Umang Gupta Greg Ver Steeg J. Ambite FedML 15 9 0 26 Apr 2022
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation An Xu Wenqi Li Pengfei Guo Dong Yang H. Roth Ali Hatamizadeh Can Zhao Daguang Xu Heng-Chiao Huang Ziyue Xu FedML 30 51 0 18 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 21 46 0 09 Mar 2022
Personalized Federated Learning with Exact Stochastic Gradient Descent Sotirios Nikoloutsopoulos I. Koutsopoulos Michalis K. Titsias FedML 25 9 0 20 Feb 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 22 20 0 12 Feb 2022
TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data Isha Garg M. Nagaraj Kaushik Roy FedML 21 1 0 21 Jan 2022
Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising Weijie Zhao Xuewu Jiao Mingqing Hu Xiaoyun Li X. Zhang Ping Li 3DV 29 8 0 05 Jan 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin R. L. Jin Tianbao Yang 39 46 0 07 Dec 2021
What Do We Mean by Generalization in Federated Learning? Honglin Yuan Warren Morningstar Lin Ning K. Singhal OOD FedML 32 71 0 27 Oct 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 36 83 0 26 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 16 0 0 01 Oct 2021
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions Arsalan Sharifnassab Saber Salehkaleybar S. J. Golestani FedML 6 0 0 19 Aug 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 17 106 0 26 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 184 411 0 14 Jul 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 24 288 0 11 Jun 2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Xinran Gu Kaixuan Huang Jingzhao Zhang Longbo Huang FedML 22 95 0 08 Jun 2021
Local Adaptivity in Federated Learning: Convergence and Consistency Jianyu Wang Zheng Xu Zachary Garrett Zachary B. Charles Luyang Liu Gauri Joshi FedML 24 39 0 04 Jun 2021
Learning from History for Byzantine Robust Optimization Sai Praneeth Karimireddy Lie He Martin Jaggi FedML AAML 22 173 0 18 Dec 2020