MARINA: Faster Non-Convex Distributed Learning with Compression

15 February 2021

Eduard A. Gorbunov

Konstantin Burlachenko

Zhize Li

Peter Richtárik

ArXiv PDF HTML

Papers citing "MARINA: Faster Non-Convex Distributed Learning with Compression"

50 / 74 papers shown

Title
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 28 0 0 12 May 2025
BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems Konstantin Burlachenko Peter Richtárik AI4CE 44 0 0 18 Mar 2025
Robust Federated Learning with Global Sensitivity Estimation for Financial Risk Management Lei Zhao Lin Cai Wu-Sheng Lu FedML 78 0 0 24 Feb 2025
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 77 1 0 22 Dec 2024
Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similarity Dmitry Bylinkin Aleksandr Beznosikov 72 1 0 21 Dec 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 64 0 0 26 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 30 0 0 11 Nov 2024
On the Convergence of FedProx with Extrapolation and Inexact Prox Hanmin Li Peter Richtárik FedML 27 0 0 02 Oct 2024
Distributed Difference of Convex Optimization Vivek Khatana M. Salapaka 25 0 0 23 Jul 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 33 1 0 25 Mar 2024
Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction Yury Demidovich Grigory Malinovsky Peter Richtárik 50 2 0 11 Mar 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat A. Maranjyan Peter Richtárik 39 3 0 07 Mar 2024
Optimal Data Splitting in Distributed Optimization for Machine Learning Daniil Medyakov Gleb Molodtsov Aleksandr Beznosikov Alexander Gasnikov 18 1 0 15 Jan 2024
Activations and Gradients Compression for Model-Parallel Training Mikhail Rudakov Aleksandr Beznosikov Yaroslav Kholodov Alexander Gasnikov 18 1 0 15 Jan 2024
Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization Pengchao Han Shiqiang Wang Yang Jiao Jianwei Huang FedML 19 5 0 20 Dec 2023
Federated Learning is Better with Non-Homomorphic Encryption Konstantin Burlachenko Abdulmajeed Alrowithi Fahad Ali Albalawi Peter Richtárik FedML 32 6 0 04 Dec 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 35 5 0 15 Oct 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 32 1 0 23 Sep 2023
AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy Zihao Zhao Yuzhu Mao Zhenpeng Shi Yang Liu Tian-Shing Lan Wenbo Ding Xiaoping Zhang 13 9 0 01 Aug 2023
Towards a Better Theoretical Understanding of Independent Subnetwork Training Egor Shulgin Peter Richtárik AI4CE 26 6 0 28 Jun 2023
Adaptive Compression in Federated Learning via Side Information Berivan Isik Francesco Pase Deniz Gunduz Sanmi Koyejo Tsachy Weissman M. Zorzi FedML 23 9 0 22 Jun 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 23 2 0 24 May 2023
Momentum Provably Improves Error Feedback! Ilyas Fatkhullin A. Tyurin Peter Richtárik 26 19 0 24 May 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression Boyue Li Yuejie Chi 21 12 0 17 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 26 7 0 12 May 2023
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features Aleksandr Beznosikov David Dobre Gauthier Gidel 25 5 0 23 Apr 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression Avetik G. Karagulyan Peter Richtárik FedML 21 6 0 08 Mar 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 21 10 0 15 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning Chanho Park Namyoon Lee FedML 14 3 0 15 Feb 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 21 18 0 01 Feb 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 22 17 0 16 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$ -Means Clustering Lingxiao Huang Zhize Li Jialin Sun Haoyu Zhao FedML 29 9 0 26 Oct 2022
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression Kaja Gruntkowska A. Tyurin Peter Richtárik 38 21 0 30 Sep 2022
Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge Sara Babakniya Souvik Kundu Saurav Prakash Yue Niu Salman Avestimehr FedML 13 9 0 27 Aug 2022
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 34 6 0 22 Aug 2022
Fast Heterogeneous Federated Learning with Hybrid Client Selection Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 27 12 0 10 Aug 2022
Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox Abdurakhmon Sadiev D. Kovalev Peter Richtárik 15 20 0 08 Jul 2022
Communication-Efficient Federated Learning With Data and Client Heterogeneity Hossein Zakerinia Shayan Talaei Giorgi Nadiradze Dan Alistarh FedML 19 7 0 20 Jun 2022
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression Zhize Li Haoyu Zhao Boyue Li Yuejie Chi FedML 22 41 0 20 Jun 2022
Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalities Aleksandr Beznosikov Alexander Gasnikov 11 10 0 19 Jun 2022
Anchor Sampling for Federated Learning with Partial Client Participation Feijie Wu Song Guo Zhihao Qu Shiqi He Ziming Liu Jing Gao FedML 25 12 0 13 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression Xinmeng Huang Yiming Chen W. Yin Kun Yuan 25 25 0 08 Jun 2022
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation Rustem Islamov Xun Qian Slavomír Hanzely M. Safaryan Peter Richtárik 30 16 0 07 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 11 11 0 02 Jun 2022
Federated Learning with a Sampling Algorithm under Isoperimetry Lukang Sun Adil Salim Peter Richtárik FedML 8 7 0 02 Jun 2022
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top Eduard A. Gorbunov Samuel Horváth Peter Richtárik Gauthier Gidel AAML 19 0 0 01 Jun 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 64 13 0 26 May 2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning Samuel Horváth Maziar Sanjabi Lin Xiao Peter Richtárik Michael G. Rabbat FedML 25 21 0 27 Apr 2022
Privacy-Aware Compression for Federated Data Analysis Kamalika Chaudhuri Chuan Guo Michael G. Rabbat FedML 25 27 0 15 Mar 2022