On the convergence properties of a $K$ -step averaging stochastic gradient descent algorithm for nonconvex optimization

3 August 2017

Papers citing "On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization"

50 / 141 papers shown

Title
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 38 0 0 08 Jan 2025
Generative AI-Powered Plugin for Robust Federated Learning in Heterogeneous IoT Networks Youngjoon Lee J. Gong Joonhyuk Kang 47 0 0 31 Oct 2024
Riemannian Federated Learning via Averaging Gradient Stream Zhenwei Huang Wen Huang Pratik Jawanpuria Bamdev Mishra FedML 24 1 0 11 Sep 2024
Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data Ji Liu Juncheng Jia Hong Zhang Yuhui Yun Leye Wang Yang Zhou H. Dai Dejing Dou FedML 30 5 0 11 Aug 2024
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 38 1 0 20 Jun 2024
ACCO: Accumulate while you Communicate, Hiding Communications in Distributed LLM Training Adel Nabli Louis Fournier Pierre Erbacher Louis Serrano Eugene Belilovsky Edouard Oyallon FedML 46 1 0 03 Jun 2024
Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging Michail Theologitis Georgios Frangias Georgios Anestis V. Samoladas Antonios Deligiannakis FedML 24 0 0 31 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 29 6 0 09 Apr 2024
Approximate Gradient Coding for Privacy-Flexible Federated Learning with Non-IID Data Okko Makkonen Sampo Niemelä Camilla Hollanti Serge Kas Hanna 27 0 0 04 Apr 2024
AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks Zhengyi Lin Guanqiao Qu Wei Wei Xianhao Chen Kin K. Leung 48 48 0 19 Mar 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 19 56 0 18 Mar 2024
Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients Xiaolu Wang Zijian Li Shi Jin Jun Zhang FedML 21 3 0 17 Feb 2024
Layered Randomized Quantization for Communication-Efficient and Privacy-Preserving Distributed Learning Guangfeng Yan Tan Li Tian-Shing Lan Kui Wu Linqi Song 19 6 0 12 Dec 2023
Robust softmax aggregation on blockchain based federated learning with convergence guarantee Huiyu Wu Diego Klabjan FedML 30 2 0 13 Nov 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 48 1 0 22 Oct 2023
Over-the-Air Federated Learning and Optimization Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Wei-Neng Chen Khaled B. Letaief FedML 23 11 0 16 Oct 2023
Based on What We Can Control Artificial Neural Networks Cheng Kang Xujing Yao 13 0 0 09 Oct 2023
Mitigating Group Bias in Federated Learning for Heterogeneous Devices Khotso Selialia Yasra Chandio Fatima M. Anwar FedML 22 2 0 13 Sep 2023
Federated Learning Robust to Byzantine Attacks: Achieving Zero Optimality Gap Shiyuan Zuo Rongfei Fan Han Hu Ningsong Zhang Shiming Gong FedML 13 2 0 21 Aug 2023
Mean Estimation with User-level Privacy under Data Heterogeneity Rachel Cummings Vitaly Feldman Audra McMillan Kunal Talwar FedML 10 25 0 28 Jul 2023
Momentum Benefits Non-IID Federated Learning Simply and Provably Ziheng Cheng Xinmeng Huang Pengfei Wu Kun Yuan FedML 29 16 0 28 Jun 2023
Federated Composite Saddle Point Optimization Site Bai Brian Bullins FedML 30 0 0 25 May 2023
Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimization Zhangyu Chang Seyyedali Hosseinalipour M. Chiang Christopher G. Brinton 24 3 0 22 May 2023
Hierarchical Weight Averaging for Deep Neural Networks Xiaozhe Gu Zixun Zhang Yuncheng Jiang Tao Luo Ruimao Zhang Shuguang Cui Zhuguo Li 16 5 0 23 Apr 2023
Making Batch Normalization Great in Federated Deep Learning Jike Zhong Hong-You Chen Wei-Lun Chao FedML 21 9 0 12 Mar 2023
When Do Curricula Work in Federated Learning? Saeed Vahidian Sreevatsank Kadaveru Woo-Ram Baek Weijia Wang Vyacheslav Kungurtsev C. L. P. Chen M. Shah Bill Lin FedML 32 11 0 24 Dec 2022
Multi-Job Intelligent Scheduling with Cross-Device Federated Learning Ji Liu Juncheng Jia Beichen Ma Chen Zhou Jingbo Zhou Yang Zhou H. Dai Dejing Dou FedML 30 23 0 24 Nov 2022
Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization Yueqi Xie Weizhong Zhang Renjie Pi Fangzhao Wu Qifeng Chen Xing Xie Sunghun Kim FedML 18 7 0 10 Nov 2022
Communication-Efficient Local SGD with Age-Based Worker Selection Feng Zhu Jingjing Zhang Xin Eric Wang 18 1 0 31 Oct 2022
Fast-Convergent Federated Learning via Cyclic Aggregation Youngjoon Lee Sangwoo Park Joonhyuk Kang FedML 28 5 0 29 Oct 2022
Scaling up Stochastic Gradient Descent for Non-convex Optimisation S. Mohamad H. Alamri A. Bouchachia 36 3 0 06 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Eric Wang 26 3 0 06 Oct 2022
$SAGDA: Achieving $\mathcal{O}(ε^{-2})$ Communication Complexity in Federated Min-Max Learning$ SAGDA: Achieving $\mathcal{O}(ε^{-2})$ Communication Complexity in Federated Min-Max Learning Haibo Yang Zhuqing Liu Xin Zhang Jia-Wei Liu FedML 25 0 0 02 Oct 2022
Personalized Federated Learning with Communication Compression El Houcine Bergou Konstantin Burlachenko Aritra Dutta Peter Richtárik FedML 72 9 0 12 Sep 2022
Flexible Vertical Federated Learning with Heterogeneous Parties Timothy Castiglia Shiqiang Wang S. Patterson FedML 22 33 0 26 Aug 2022
Fast Heterogeneous Federated Learning with Hybrid Client Selection Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 27 12 0 10 Aug 2022
Multi-Model Federated Learning with Provable Guarantees Neelkamal Bhuyan Sharayu Moharir Gauri Joshi FedML 18 14 0 09 Jul 2022
On the Importance and Applicability of Pre-Training for Federated Learning Hong-You Chen Cheng-Hao Tu Zi-hua Li Hang Shen Wei-Lun Chao FedML 17 77 0 23 Jun 2022
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data Jianyu Wang Rudrajit Das Gauri Joshi Satyen Kale Zheng Xu Tong Zhang FedML 28 38 0 09 Jun 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 24 75 0 27 May 2022
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks Mingrui Liu Zhenxun Zhuang Yunwei Lei Chunyang Liao 22 16 0 10 May 2022
FedGiA: An Efficient Hybrid Algorithm for Federated Learning Shenglong Zhou Geoffrey Ye Li FedML 28 16 0 03 May 2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning Samuel Horváth Maziar Sanjabi Lin Xiao Peter Richtárik Michael G. Rabbat FedML 25 21 0 27 Apr 2022
Distributed Evolution Strategies for Black-box Stochastic Optimization Xiaoyu He Zibin Zheng Chuan Chen Yuren Zhou Chuan Luo Qingwei Lin 19 4 0 09 Apr 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 27 3 0 22 Mar 2022
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 14 2 0 13 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 19 46 0 09 Mar 2022
Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning Wenzhi Fang Ziyi Yu Yuning Jiang Yuanming Shi Colin N. Jones Yong Zhou FedML 76 56 0 24 Jan 2022
Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising Weijie Zhao Xuewu Jiao Mingqing Hu Xiaoyun Li X. Zhang Ping Li 3DV 27 8 0 05 Jan 2022
FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 17 25 0 22 Dec 2021