Sparsified SGD with Memory

20 September 2018

Papers citing "Sparsified SGD with Memory"

50 / 134 papers shown

Title
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 26 0 0 01 May 2025
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao Weigang Wu FedML 39 0 0 28 Apr 2025
FedFetch: Faster Federated Learning with Adaptive Downstream Prefetching Qifan Yan Andrew Liu Shiqi He Mathias Lécuyer Ivan Beschastnikh FedML 36 0 0 21 Apr 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 46 0 0 11 Mar 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 46 1 0 28 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 31 2 0 21 Oct 2024
Collaborative and Efficient Personalization with Mixtures of Adaptors Abdulla Jasem Almansoori Samuel Horváth Martin Takáč FedML 44 2 0 04 Oct 2024
Communication-efficient Vertical Federated Learning via Compressed Error Feedback Pedro Valdeira João Xavier Cláudia Soares Yuejie Chi FedML 39 4 0 20 Jun 2024
VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections Roy Miles Pradyumna Reddy Ismail Elezi Jiankang Deng VLM 32 3 0 28 May 2024
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization Zhe Li Bicheng Ying Zidong Liu Haibo Yang Haibo Yang FedML 59 3 0 24 May 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 42 0 0 22 May 2024
PAFedFV: Personalized and Asynchronous Federated Learning for Finger Vein Recognition Hengyu Mu Jian Guo Chong Han Lijuan Sun FedML 24 5 0 20 Apr 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 38 1 0 25 Mar 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes Xiaoxin Su Yipeng Zhou Laizhong Cui John C. S. Lui Jiangchuan Liu FedML 27 1 0 06 Feb 2024
Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillation Vasileios Tsouvalas Aaqib Saeed T. Ozcelebi N. Meratnia FedML 34 6 0 25 Jan 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Kimad: Adaptive Gradient Compression with Bandwidth Awareness Jihao Xin Ivan Ilin Shunkang Zhang Marco Canini Peter Richtárik 32 2 0 13 Dec 2023
AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term Memory Haifeng Wen Hong Xing Osvaldo Simeone 32 0 0 25 Oct 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 26 0 0 19 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 37 5 0 15 Oct 2023
Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks Xiaonan Liu T. Ratnarajah M. Sellathurai Yonina C. Eldar 29 4 0 04 Sep 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 28 0 0 01 Aug 2023
An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning Cheng Yang Xue Yang Dongxian Wu Xiaohu Tang FedML 27 0 0 21 Jun 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 27 10 0 30 May 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 23 2 0 24 May 2023
Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression Boyue Li Yuejie Chi 21 12 0 17 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 28 7 0 12 May 2023
Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy Kang Wei Jun Li Chuan Ma Ming Ding Feng Shu Haitao Zhao Wen Chen Hongbo Zhu FedML 25 4 0 09 Apr 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression Avetik G. Karagulyan Peter Richtárik FedML 26 6 0 08 Mar 2023
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities Aleksandr Beznosikov Martin Takáč Alexander Gasnikov 23 10 0 15 Feb 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 11 4 0 23 Jan 2023
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes Konstantin Mishchenko Slavomír Hanzely Peter Richtárik FedML 24 5 0 17 Jan 2023
FedCliP: Federated Learning with Client Pruning Beibei Li Zerui Shao Ao Liu Peiran Wang FedML 37 1 0 17 Jan 2023
CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergence Kun-Yen Huang Shin-Yi Pu 30 9 0 14 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 19 4 0 06 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao L. Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 21 5 0 08 Dec 2022
GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning Shiqi He Qifan Yan Feijie Wu Lanjun Wang Mathias Lécuyer Ivan Beschastnikh FedML 36 7 0 03 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges Enrique Tomás Martínez Beltrán Mario Quiles Pérez Pedro Miguel Sánchez Sánchez Sergio López Bernal Gérome Bovet M. Pérez Gregorio Martínez Pérez Alberto Huertas Celdrán FedML 21 221 0 15 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 21 12 0 31 Oct 2022
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 14 6 0 31 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
Adaptive Top-K in SGD for Communication-Efficient Distributed Learning Mengzhe Ruan Guangfeng Yan Yuanzhang Xiao Linqi Song Weitao Xu 29 3 0 24 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 25 0 0 14 Oct 2022
Downlink Compression Improves TopK Sparsification William Zou H. Sterck Jun Liu 16 0 0 30 Sep 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 11 23 0 12 Aug 2022
Fast Heterogeneous Federated Learning with Hybrid Client Selection Guangyuan Shen D. Gao Duanxiao Song Libin Yang Xukai Zhou Shirui Pan W. Lou Fang Zhou FedML 27 12 0 10 Aug 2022
QC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM Ping Xu Yue Wang Xiang Chen Zhi Tian 16 2 0 04 Aug 2022