Tighter Theory for Local SGD on Identical and Heterogeneous Data

10 September 2019

Papers citing "Tighter Theory for Local SGD on Identical and Heterogeneous Data"

50 / 95 papers shown

Title
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 30 0 0 12 May 2025
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 55 20 0 20 Apr 2025
Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling Feng Zhu Aritra Mitra Robert W. Heath FedML 36 0 0 15 Apr 2025
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models Jialiang Cheng Ning Gao Yun Yue Zhiling Ye Jiadi Jiang Jian Sha OffRL 77 0 0 10 Dec 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 66 0 0 26 Nov 2024
Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information Emanuel Buttaci Giuseppe Carlo Calafiore FedML 27 0 0 07 Oct 2024
On the Convergence of a Federated Expectation-Maximization Algorithm Zhixu Tao Rajita Chandak Sanjeev R. Kulkarni FedML 30 0 0 11 Aug 2024
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization Jiayi Wang Shiqiang Wang Rong-Rong Chen Mingyue Ji FedML 30 1 0 22 Jul 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 46 6 0 19 May 2024
Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method Elissa Mhanna Mohamad Assaad 41 1 0 30 Jan 2024
Kimad: Adaptive Gradient Compression with Bandwidth Awareness Jihao Xin Ivan Ilin Shunkang Zhang Marco Canini Peter Richtárik 32 2 0 13 Dec 2023
ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations Xinpeng Ling Jie Fu Kuncan Wang Haitao Liu Zhili Chen FedML 26 2 0 21 Aug 2023
Benchmarking Algorithms for Federated Domain Generalization Ruqi Bai S. Bagchi David I. Inouye FedML 34 12 0 11 Jul 2023
Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates Siqi Zhang S. Choudhury Sebastian U. Stich Nicolas Loizou FedML 19 3 0 08 Jun 2023
Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging Jiamian Wang Zong-Jhe Wu Yulun Zhang Xin Yuan Tao R. Lin Zhiqiang Tao FedML 51 3 0 01 Jun 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 27 10 0 30 May 2023
DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regime Jongho Park Jinchao Xu FedML 47 1 0 17 May 2023
Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling Nicolò Dal Fabbro A. Mitra George J. Pappas FedML 35 12 0 14 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 43 2 0 09 Apr 2023
Federated Gradient Matching Pursuit Halyun Jeong Deanna Needell Jing Qin FedML 35 1 0 20 Feb 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 26 10 0 15 Feb 2023
Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity Han Wang A. Mitra Hamed Hassani George J. Pappas James Anderson FedML 26 21 0 04 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 40 6 0 30 Jan 2023
Graph Federated Learning for CIoT Devices in Smart Home Applications Arash Rasti-Meymandi S. M. Sheikholeslami J. Abouei Konstantinos N. Plataniotis FedML 19 18 0 29 Dec 2022
When Do Curricula Work in Federated Learning? Saeed Vahidian Sreevatsank Kadaveru Woo-Ram Baek Weijia Wang Vyacheslav Kungurtsev C. L. P. Chen M. Shah Bill Lin FedML 37 11 0 24 Dec 2022
Local Model Reconstruction Attacks in Federated Learning and their Uses Ilias Driouich Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML FedML 29 2 0 28 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability Zhao-quan Song Yitan Wang Zheng Yu Licheng Zhang FedML 23 28 0 15 Oct 2022
On the Performance of Gradient Tracking with Local Updates Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe 35 19 0 10 Oct 2022
FedMT: Federated Learning with Mixed-type Labels Qiong Zhang Jing Peng Xin Zhang A. Talhouk Gang Niu Xiaoxiao Li FedML 51 0 0 05 Oct 2022
PersA-FL: Personalized Asynchronous Federated Learning Taha Toghani Soomin Lee César A. Uribe FedML 34 6 0 03 Oct 2022
Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation Taha Toghani César A. Uribe FedML 35 14 0 03 Oct 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 29 1 0 25 Sep 2022
HammingMesh: A Network Topology for Large-Scale Deep Learning Torsten Hoefler Tommaso Bonato Daniele De Sensi Salvatore Di Girolamo Shigang Li Marco Heddes Jon Belk Deepak Goel Miguel Castro Steve Scott 3DH GNN AI4CE 26 20 0 03 Sep 2022
Fast Heterogeneous Federated Learning with Hybrid Client Selection Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 29 12 0 10 Aug 2022
Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds A. Mitra Arman Adibi George J. Pappas Hamed Hassani 44 6 0 06 Jun 2022
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning Liam Collins Hamed Hassani Aryan Mokhtari Sanjay Shakkottai FedML 26 75 0 27 May 2022
Personalized Federated Learning with Server-Side Information Jaehun Song Min Hwan Oh Hyung-Sin Kim FedML 33 8 0 23 May 2022
Tighter Regret Analysis and Optimization of Online Federated Learning Dohyeok Kwon Jonghwan Park Songnam Hong 26 11 0 13 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 22 10 0 08 May 2022
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction Junyi Li Feihu Huang Heng-Chiao Huang FedML 19 27 0 03 May 2022
SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networks Won Joon Yun Yunseok Kwak Hankyul Baek Soyi Jung Mingyue Ji M. Bennis Jihong Park Joongheon Kim 13 16 0 26 Mar 2022
FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction Liang Gao H. Fu Li Li Yingwen Chen Minghua Xu Chengzhong Xu FedML 21 242 0 22 Mar 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 27 3 0 22 Mar 2022
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning Lin Zhang Li Shen Liang Ding Dacheng Tao Ling-Yu Duan FedML 28 252 0 17 Mar 2022
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Pranay Sharma Rohan Panda Gauri Joshi P. Varshney FedML 21 46 0 09 Mar 2022
On the Convergence of Clustered Federated Learning Ma Jie Guodong Long Tianyi Zhou Jing Jiang Chengqi Zhang FedML 33 46 0 13 Feb 2022
Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning Wenzhi Fang Ziyi Yu Yuning Jiang Yuanming Shi Colin N. Jones Yong Zhou FedML 76 56 0 24 Jan 2022
Towards Federated Learning on Time-Evolving Heterogeneous Data Yongxin Guo Tao R. Lin Xiaoying Tang FedML 14 30 0 25 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 36 16 0 05 Dec 2021