An improved convergence analysis for decentralized online stochastic non-convex optimization

10 August 2020

Papers citing "An improved convergence analysis for decentralized online stochastic non-convex optimization"

50 / 54 papers shown

Title
Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum Yuan Zhou Xinli Shi Xuelong Li Jiachen Zhong G. Wen Jinde Cao FedML 50 0 0 17 Apr 2025
Decentralized Federated Domain Generalization with Style Sharing: A Formal Modeling and Convergence Analysis Shahryar Zehtabi Dong-Jun Han Seyyedali Hosseinalipour Christopher G. Brinton FedML AI4CE 50 0 0 08 Apr 2025
Federated Learning for Diffusion Models Zihao Peng Xijun Wang Shengbo Chen Hong Rao Cong Shen DiffM FedML 55 0 0 09 Mar 2025
Faster Adaptive Decentralized Learning Algorithms Feihu Huang Jianyu Zhao 56 0 0 19 Aug 2024
Nonlinear Perturbation-based Non-Convex Optimization over Time-Varying Networks Mohammadreza Doostmohammadian Zulfiya R. Gabidullina Hamid R. Rabiee 48 7 0 05 Aug 2024
Accelerating Distributed Optimization: A Primal-Dual Perspective on Local Steps Junchi Yang Murat Yildirim Qiu Feng 50 0 0 02 Jul 2024
The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity Tongle Wu Ying Sun 44 0 0 23 Mar 2024
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions Siyuan Zhang Nachuan Xiao Xin Liu 61 1 0 18 Mar 2024
An Accelerated Distributed Stochastic Gradient Method with Momentum Kun-Yen Huang Shi Pu Angelia Nedić 35 8 0 15 Feb 2024
Problem-Parameter-Free Decentralized Nonconvex Stochastic Optimization Jiaxiang Li Xuxing Chen Shiqian Ma Mingyi Hong ODL 21 2 0 13 Feb 2024
Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs Ziqin Chen Yongqiang Wang FedML 24 2 0 24 Oct 2023
Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization Zhenwei Lin Jingfan Xia Qi Deng Luo Luo 31 3 0 18 Oct 2023
Federated Multi-Level Optimization over Decentralized Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang AI4CE 28 0 0 10 Oct 2023
Decentralized Local Updates with Dual-Slow Estimation and Momentum-based Variance-Reduction for Non-Convex Optimization Kangyang Luo Kunkun Zhang Sheng Zhang Xiang Li Ming Gao 30 2 0 17 Jul 2023
Momentum Benefits Non-IID Federated Learning Simply and Provably Ziheng Cheng Xinmeng Huang Pengfei Wu Kun Yuan FedML 35 17 0 28 Jun 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 24 4 0 21 Jun 2023
DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm Lisang Ding Kexin Jin Bicheng Ying Kun Yuan W. Yin 11 9 0 01 Jun 2023
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much? Yutong He Xinmeng Huang Kun Yuan 31 9 0 25 May 2023
Distributed Learning over Networks with Graph-Attention-Based Personalization Zhuojun Tian Zhaoyang Zhang Zhaohui Yang Richeng Jin H. Dai GNN FedML 25 6 0 22 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 36 7 0 12 May 2023
PRECISION: Decentralized Constrained Min-Max Learning with Low Communication and Sample Complexities Zhuqing Liu Xin Zhang Songtao Lu Jia-Wei Liu 40 7 0 05 Mar 2023
A Unified Momentum-based Paradigm of Decentralized SGD for Non-Convex Models and Heterogeneous Data Haizhou Du Chengdong Ni 22 1 0 01 Mar 2023
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization Tesi Xiao Xuxing Chen Krishnakumar Balasubramanian Saeed Ghadimi 36 10 0 20 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 43 6 0 30 Jan 2023
CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergence Kun-Yen Huang Shin-Yi Pu 38 9 0 14 Jan 2023
Optimal Complexity in Non-Convex Decentralized Learning over Time-Varying Networks Xinmeng Huang Kun Yuan 29 6 0 01 Nov 2022
Communication-Efficient Topologies for Decentralized Learning with $O(1)$ Consensus Rate Zhuoqing Song Weijian Li Kexin Jin Lei Shi Ming Yan W. Yin Kun Yuan 11 31 0 14 Oct 2022
Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization Kun Yuan Xinmeng Huang Yiming Chen Xiaohan Zhang Yingya Zhang Pan Pan 34 17 0 14 Oct 2022
On the Performance of Gradient Tracking with Local Updates Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe 43 19 0 10 Oct 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 42 11 0 15 Sep 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 30 14 0 17 Aug 2022
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang 40 42 0 22 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression Xinmeng Huang Yiming Chen W. Yin Kun Yuan 30 25 0 08 Jun 2022
Heavy-Tail Phenomenon in Decentralized SGD Mert Gurbuzbalaban Yuanhan Hu Umut Simsekli Kun Yuan Lingjiong Zhu 38 8 0 13 May 2022
Distributed Finite-Sum Constrained Optimization subject to Nonlinearity on the Node Dynamics Mohammadreza Doostmohammadian Maria Vrakopoulou Alireza Aghasi Themistoklis Charalambous 15 4 0 28 Mar 2022
Variance reduced stochastic optimization over directed graphs with row and column stochastic weights Muhammad I. Qureshi Ran Xin S. Kar U. Khan 16 3 0 07 Feb 2022
Distributed Random Reshuffling over Networks Kun-Yen Huang Xiao Li Andre Milzarek Shi Pu Junwen Qiu 39 11 0 31 Dec 2021
Decentralized Stochastic Proximal Gradient Descent with Variance Reduction over Time-varying Networks Xuanjie Li Yuedong Xu Jessie Hui Wang Xin Wang John C. S. Lui 12 4 0 20 Dec 2021
Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning Xiaoxiao Zhao Jinlong Lei Li Li Jie-bin Chen OffRL 20 2 0 25 Nov 2021
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning Bicheng Ying Kun Yuan Hanbin Hu Yiming Chen W. Yin FedML 39 27 0 08 Nov 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 39 83 0 26 Oct 2021
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning Sulaiman A. Alghunaim Kun Yuan 42 57 0 19 Oct 2021
An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models A. Rao Hoi-To Wai 19 0 0 09 Oct 2021
Distributed stochastic gradient tracking algorithm with variance reduction for non-convex optimization Xia Jiang Xianlin Zeng Jian Sun Jie Chen 26 14 0 28 Jun 2021
Accelerating Gossip SGD with Periodic Global Averaging Yiming Chen Kun Yuan Yingya Zhang Pan Pan Yinghui Xu W. Yin 37 41 0 19 May 2021
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD Kun Yuan Sulaiman A. Alghunaim Xinmeng Huang 31 32 0 17 May 2021
Improving the Transient Times for Distributed Stochastic Gradient Methods Kun-Yen Huang Shi Pu 29 23 0 11 May 2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training Kun Yuan Yiming Chen Xinmeng Huang Yingya Zhang Pan Pan Yinghui Xu W. Yin MoE 60 61 0 24 Apr 2021
A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization Ran Xin U. Khan S. Kar 20 39 0 12 Feb 2021
A fast randomized incremental gradient method for decentralized non-convex optimization Ran Xin U. Khan S. Kar 19 33 0 07 Nov 2020