Error Feedback Fixes SignSGD and other Gradient Compression Schemes

28 January 2019

Sai Praneeth Karimireddy

Papers citing "Error Feedback Fixes SignSGD and other Gradient Compression Schemes"

50 / 87 papers shown

Title
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao Weigang Wu FedML 39 0 0 28 Apr 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 49 0 0 11 Mar 2025
On the Byzantine Fault Tolerance of signSGD with Majority Vote Emanuele Mengoli Luzius Moll Virgilio Strozzi El-Mahdi El-Mhamdi AAML FedML 60 0 0 26 Feb 2025
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 66 0 0 26 Nov 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 48 1 0 28 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 31 2 0 21 Oct 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
Communication-efficient Vertical Federated Learning via Compressed Error Feedback Pedro Valdeira João Xavier Cláudia Soares Yuejie Chi FedML 39 4 0 20 Jun 2024
BOLD: Boolean Logic Deep Learning Van Minh Nguyen Cristian Ocampo Aymen Askri Louis Leconte Ba-Hien Tran AI4CE 37 0 0 25 May 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 45 0 0 22 May 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 40 1 0 25 Mar 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network Student Member Ieee Hongda Wu F. I. C. V. Ping Wang Aswartha Narayana FedML 44 1 0 16 Nov 2023
AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term Memory Haifeng Wen Hong Xing Osvaldo Simeone 32 0 0 25 Oct 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 18 1 0 24 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 40 5 0 15 Oct 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 23 6 0 01 Aug 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 30 0 0 01 Aug 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 27 10 0 30 May 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 23 2 0 24 May 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods Junchi Yang Xiang Li Ilyas Fatkhullin Niao He 34 15 0 21 May 2023
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy Zhenxiao Zhang Yuanxiong Guo Yuguang Fang Yanmin Gong 28 4 0 15 Apr 2023
A Comprehensive Survey of Dataset Distillation Shiye Lei Dacheng Tao DD 31 87 0 13 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
Fast Adaptive Federated Bilevel Optimization Feihu Huang FedML 20 7 0 02 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 24 12 0 31 Oct 2022
Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$ -Means Clustering Lingxiao Huang Zhize Li Jialin Sun Haoyu Zhao FedML 33 9 0 26 Oct 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 24 6 0 25 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 25 0 0 14 Oct 2022
Downlink Compression Improves TopK Sparsification William Zou H. Sterck Jun Liu 18 0 0 30 Sep 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 32 46 0 23 Aug 2022
Efficient-Adam: Communication-Efficient Distributed Adam Congliang Chen Li Shen Wei Liu Z. Luo 23 19 0 28 May 2022
Communication-Efficient Adaptive Federated Learning Yujia Wang Lu Lin Jinghui Chen FedML 21 70 0 05 May 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 17 10 0 16 Apr 2022
Convert, compress, correct: Three steps toward communication-efficient DNN training Zhongzhu Chen Eduin E. Hernandez Yu-Chih Huang Stefano Rini 15 0 0 17 Mar 2022
Linear Stochastic Bandits over a Bit-Constrained Channel A. Mitra Hamed Hassani George J. Pappas 34 8 0 02 Mar 2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 19 47 0 15 Feb 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
BEER: Fast $O(1/T)$ Rate for Decentralized Nonconvex Optimization with Communication Compression Haoyu Zhao Boyue Li Zhize Li Peter Richtárik Yuejie Chi 19 48 0 31 Jan 2022
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 18 0 0 08 Dec 2021
Distributed Adaptive Learning Under Communication Constraints Marco Carpentiero Vincenzo Matta A. H. Sayed 22 17 0 03 Dec 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 24 14 0 01 Nov 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 28 46 0 11 Oct 2021
Comfetch: Federated Learning of Large Networks on Constrained Clients via Sketching Tahseen Rabbani Brandon Yushan Feng Marco Bornstein Kyle Rui Sang Yifan Yang Arjun Rajkumar A. Varshney Furong Huang FedML 51 2 0 17 Sep 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 22 45 0 19 Aug 2021
Decentralized Composite Optimization with Compression Yao Li Xiaorui Liu Jiliang Tang Ming Yan Kun Yuan 19 9 0 10 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 43 54 0 02 Aug 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning Kaan Ozkara Navjot Singh Deepesh Data Suhas Diggavi FedML MQ 24 56 0 29 Jul 2021