Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization

26 January 2022

Grigory Malinovsky

Konstantin Mishchenko

Peter Richtárik

FedML

ArXiv PDF HTML

Papers citing "Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization"

21 / 21 papers shown

Title
Task Arithmetic Through The Lens Of One-Shot Federated Learning Zhixu Tao I. Mason Sanjeev R. Kulkarni Xavier Boix MoMe FedML 84 3 0 27 Nov 2024
Loss Landscape Characterization of Neural Networks without Over-Parametrization Rustem Islamov Niccolò Ajroldi Antonio Orvieto Aurelien Lucchi 35 4 0 16 Oct 2024
FedECADO: A Dynamical System Model of Federated Learning Aayushya Agarwal Gauri Joshi L. Pileggi FedML 23 0 0 13 Oct 2024
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 32 2 0 10 Oct 2024
Towards Hyper-parameter-free Federated Learning Geetika Drishya Uniyal Bapi Chatterjee FedML 52 0 0 30 Aug 2024
Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning Kai Yi Timur Kharisov Igor Sokolov Peter Richtárik FedML 32 2 0 03 Jun 2024
Towards Fairness in Provably Communication-Efficient Federated Recommender Systems Kirandeep Kaur Sujit Gujar Shweta Jain FedML 48 0 0 03 May 2024
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms Rustem Islamov M. Safaryan Dan Alistarh FedML 26 12 0 31 Oct 2023
Improving Federated Aggregation with Deep Unfolding Networks S. Nanayakkara Shiva Raj Pokhrel Gang Li FedML 31 0 0 30 Jun 2023
Towards a Better Theoretical Understanding of Independent Subnetwork Training Egor Shulgin Peter Richtárik AI4CE 28 6 0 28 Jun 2023
Improving Accelerated Federated Learning with Compression and Importance Sampling Michal Grudzieñ Grigory Malinovsky Peter Richtárik FedML 35 8 0 05 Jun 2023
On Convergence of Incremental Gradient for Non-Convex Smooth Functions Anastasia Koloskova N. Doikov Sebastian U. Stich Martin Jaggi 36 2 0 30 May 2023
Federated Learning with Regularized Client Participation Grigory Malinovsky Samuel Horváth Konstantin Burlachenko Peter Richtárik FedML 31 13 0 07 Feb 2023
Coordinating Distributed Example Orders for Provably Accelerated Training A. Feder Cooper Wentao Guo Khiem Pham Tiancheng Yuan Charlie F. Ruan Yucheng Lu Chris De Sa 38 6 0 02 Feb 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 40 6 0 30 Jan 2023
FedExP: Speeding Up Federated Averaging via Extrapolation Divyansh Jhunjhunwala Shiqiang Wang Gauri Joshi FedML 19 52 0 23 Jan 2023
Convergence of ease-controlled Random Reshuffling gradient Algorithms under Lipschitz smoothness R. Seccia Corrado Coppola G. Liuzzi L. Palagi 26 2 0 04 Dec 2022
Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox Abdurakhmon Sadiev D. Kovalev Peter Richtárik 27 20 0 08 Jul 2022
Federated Optimization Algorithms with Random Reshuffling and Gradient Compression Abdurakhmon Sadiev Grigory Malinovsky Eduard A. Gorbunov Igor Sokolov Ahmed Khaled Konstantin Burlachenko Peter Richtárik FedML 16 21 0 14 Jun 2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning Samuel Horváth Maziar Sanjabi Lin Xiao Peter Richtárik Michael G. Rabbat FedML 25 21 0 27 Apr 2022
Bolstering Stochastic Gradient Descent with Model Building Ş. Birbil Özgür Martin Gönenç Onay Figen Oztoprak ODL 14 1 0 13 Nov 2021