v1v2v3 (latest)

MARINA: Faster Non-Convex Distributed Learning with Compression

International Conference on Machine Learning (ICML), 2021

15 February 2021

Eduard A. Gorbunov

Konstantin Burlachenko

Zhize Li

Peter Richtárik

ArXiv (abs)PDF HTML

Papers citing "MARINA: Faster Non-Convex Distributed Learning with Compression"

50 / 75 papers shown

Title
An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning Zhijing Ye Sheng Di Jiamin Wang Zhiqing Zhong Zhaorui Zhang Xiaodong Yu FedML 148 0 0 07 Nov 2025
Coded Robust Aggregation for Distributed Learning under Byzantine Attacks Chengxi Li Ming Xiao Mikael Skoglund AAML OOD 128 0 0 17 May 2025
Bant: Byzantine Antidote via Trial Function and Trust Scores Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 553 1 0 12 May 2025
BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems Konstantin Burlachenko Peter Richtárik AI4CE 184 0 0 18 Mar 2025
Robust Federated Learning with Global Sensitivity Estimation for Financial Risk Management Lei Zhao Lin Cai Wu-Sheng Lu FedML 257 2 0 24 Feb 2025
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 283 1 0 22 Dec 2024
Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data SimilarityAAAI Conference on Artificial Intelligence (AAAI), 2024 Dmitry Bylinkin Aleksandr Beznosikov 383 1 0 21 Dec 2024
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 278 0 0 26 Nov 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 412 1 0 11 Nov 2024
On the Convergence of FedProx with Extrapolation and Inexact Prox Hanmin Li Peter Richtárik FedML 257 3 0 02 Oct 2024
Distributed Difference of Convex Optimization Vivek Khatana M. Salapaka 210 0 0 23 Jul 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 191 2 0 25 Mar 2024
Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction Yury Demidovich Grigory Malinovsky Peter Richtárik 213 3 0 11 Mar 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 347 6 0 07 Mar 2024
Optimal Data Splitting in Distributed Optimization for Machine LearningDoklady. Mathematics (Dokl. Math.), 2023 Daniil Medyakov Gleb Molodtsov Aleksandr Beznosikov Alexander Gasnikov 234 4 0 15 Jan 2024
Activations and Gradients Compression for Model-Parallel TrainingDoklady. Mathematics (Dokl. Math.), 2023 Mikhail Rudakov Aleksandr Beznosikov Yaroslav Kholodov Alexander Gasnikov 323 5 0 15 Jan 2024
Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization Pengchao Han Maroun Touma Yang Jiao Jianwei Huang FedML 257 12 0 20 Dec 2023
Federated Learning is Better with Non-Homomorphic Encryption Konstantin Burlachenko Abdulmajeed Alrowithi Fahad Ali Albalawi Peter Richtárik FedML 220 6 0 04 Dec 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 300 11 0 15 Oct 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 188 1 0 23 Sep 2023
AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection StrategyIEEE Transactions on Mobile Computing (IEEE TMC), 2023 Zihao Zhao Yuzhu Mao Zhenpeng Shi Yang Liu Tian-Shing Lan Wenbo Ding Xiaoping Zhang 304 17 0 01 Aug 2023
Towards a Better Theoretical Understanding of Independent Subnetwork TrainingInternational Conference on Machine Learning (ICML), 2023 Egor Shulgin Peter Richtárik AI4CE 324 8 0 28 Jun 2023
Adaptive Compression in Federated Learning via Side InformationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Berivan Isik Francesco Pase Deniz Gunduz Sanmi Koyejo Tsachy Weissman M. Zorzi FedML 140 17 0 22 Jun 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 178 2 0 24 May 2023
Momentum Provably Improves Error Feedback!Neural Information Processing Systems (NeurIPS), 2023 Ilyas Fatkhullin Alexander Tyurin Peter Richtárik 285 37 0 24 May 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication CompressionIEEE Journal on Selected Topics in Signal Processing (JSTSP), 2023 Boyue Li Yuejie Chi 157 17 0 17 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 355 9 0 12 May 2023
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical FeaturesInternational Conference on Machine Learning (ICML), 2023 Aleksandr Beznosikov David Dobre Gauthier Gidel 208 7 0 23 Apr 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional CompressionConference on Uncertainty in Artificial Intelligence (UAI), 2023 Avetik G. Karagulyan Peter Richtárik FedML 227 6 0 08 Mar 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational InequalitiesNeural Information Processing Systems (NeurIPS), 2023 Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 260 12 0 15 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed LearningInternational Symposium on Information Theory (ISIT), 2023 Chanho Park Namyoon Lee FedML 144 6 0 15 Feb 2023
DoCoFL: Downlink Compression for Cross-Device Federated LearningInternational Conference on Machine Learning (ICML), 2023 Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 281 33 0 01 Feb 2023
Federated Learning with Flexible ControlIEEE Conference on Computer Communications (INFOCOM), 2022 Maroun Touma Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 210 20 0 16 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 188 6 0 25 Nov 2022
Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$ -Means ClusteringNeural Information Processing Systems (NeurIPS), 2022 Lingxiao Huang Zhize Li Jialin Sun Haoyu Zhao FedML 187 15 0 26 Oct 2022
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional CompressionInternational Conference on Machine Learning (ICML), 2022 Kaja Gruntkowska Alexander Tyurin Peter Richtárik 350 31 0 30 Sep 2022
Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge Sara Babakniya Souvik Kundu Saurav Prakash Yue Niu Salman Avestimehr FedML 229 15 0 27 Aug 2022
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex OptimizationJournal of machine learning research (JMLR), 2022 Zhize Li Jian Li 261 9 0 22 Aug 2022
Fast Heterogeneous Federated Learning with Hybrid Client SelectionConference on Uncertainty in Artificial Intelligence (UAI), 2022 Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 336 15 0 10 Aug 2022
Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact ProxNeural Information Processing Systems (NeurIPS), 2022 Abdurakhmon Sadiev D. Kovalev Peter Richtárik 245 23 0 08 Jul 2022
SoteriaFL: A Unified Framework for Private Federated Learning with Communication CompressionNeural Information Processing Systems (NeurIPS), 2022 Zhize Li Haoyu Zhao Boyue Li Yuejie Chi FedML 212 49 0 20 Jun 2022
Communication-Efficient Federated Learning With Data and Client HeterogeneityInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2022 Hossein Zakerinia Shayan Talaei Giorgi Nadiradze Dan Alistarh FedML 403 12 0 20 Jun 2022
Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalities Aleksandr Beznosikov Alexander Gasnikov 219 11 0 19 Jun 2022
Anchor Sampling for Federated Learning with Partial Client ParticipationInternational Conference on Machine Learning (ICML), 2022 Feijie Wu Song Guo Zhihao Qu Shiqi He Ziming Liu Jing Gao FedML 209 23 0 13 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication CompressionNeural Information Processing Systems (NeurIPS), 2022 Xinmeng Huang Yiming Chen W. Yin Kun Yuan 311 31 0 08 Jun 2022
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation Rustem Islamov Xun Qian Slavomír Hanzely M. Safaryan Peter Richtárik 167 21 0 07 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with GuaranteesNeural Information Processing Systems (NeurIPS), 2022 Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 378 18 0 02 Jun 2022
Federated Learning with a Sampling Algorithm under Isoperimetry Lukang Sun Adil Salim Peter Richtárik FedML 229 8 0 02 Jun 2022
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top Eduard A. Gorbunov Samuel Horváth Peter Richtárik Gauthier Gidel AAML 247 0 0 01 Jun 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 325 14 0 26 May 2022