v1v2v3 (latest)

signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu Wang

Kamyar Azizzadenesheli

Anima Anandkumar

FedML

ODL

ArXiv (abs)PDF HTML Github (86★)

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 592 papers shown

Title
FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications Grant Wilkins Sheng Di Jon C. Calhoun Zilinghan Li Kibaek Kim Robert Underwood Richard Mortier Franck Cappello FedML 206 9 0 20 Dec 2023
Contractive error feedback for gradient compression Bingcong Li Shuai Zheng Parameswaran Raman Anshumali Shrivastava G. Giannakis 147 0 0 13 Dec 2023
Flexible Communication for Optimal Distributed Learning over Unpredictable NetworksBigData Congress [Services Society] (BSS), 2023 S. Tyagi Martin Swany 362 2 0 05 Dec 2023
Green Edge AI: A Contemporary SurveyProceedings of the IEEE (Proc. IEEE), 2023 Yuyi Mao X. Yu Kaibin Huang Ying-Jun Angela Zhang Jun Zhang 318 49 0 01 Dec 2023
Edge AI for Internet of Energy: Challenges and PerspectivesInternet of Things (IoT), 2023 Yassine Himeur A. Sayed A. Alsalemi F. Bensaali Abbes Amira 272 48 0 28 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 218 18 0 22 Nov 2023
Examining Common Paradigms in Multi-Task Learning Cathrin Elich Lukas Kirchdorfer Jan M. Kohler Lukas Schott 208 3 0 08 Nov 2023
Blind Federated Learning via Over-the-Air q-QAMIEEE Transactions on Wireless Communications (IEEE TWC), 2023 Saeed Razavikia José Hélio da Cruz Júnior Carlo Fischione 244 9 0 07 Nov 2023
EControl: Fast Distributed Optimization with Compression and Error ControlInternational Conference on Learning Representations (ICLR), 2023 Yuan Gao Rustem Islamov Sebastian U. Stich 197 14 0 06 Nov 2023
Signal Processing Meets SGD: From Momentum to Filter Zhipeng Yao Guisong Chang Jiaqi Zhang Qi Zhang Dazhou Li Yu Zhang ODL 532 0 0 06 Nov 2023
On the accuracy and efficiency of group-wise clipping in differentially private optimization Zhiqi Bu Ruixuan Liu Yu Wang Sheng Zha George Karypis VLM 166 5 0 30 Oct 2023
Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication CompressionInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Sijin Chen Zhize Li Yuejie Chi FedML 185 5 0 29 Oct 2023
Correlation Aware Sparsified Mean Estimation Using Random ProjectionNeural Information Processing Systems (NeurIPS), 2023 Shuli Jiang Pranay Sharma Gauri Joshi 248 2 0 29 Oct 2023
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise Aleksandar Armacki Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 566 10 0 28 Oct 2023
A Spectral Condition for Feature Learning Greg Yang James B. Simon Jeremy Bernstein 265 58 0 26 Oct 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 198 6 0 26 Oct 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 256 3 0 24 Oct 2023
Projected Stochastic Gradient Descent with Quantum Annealed Binary GradientsBritish Machine Vision Conference (BMVC), 2023 Maximilian Krahn Michele Sasdelli Fengyi Yang Vladislav Golyanik Arno Solin Tat-Jun Chin Tolga Birdal MQ 381 3 0 23 Oct 2023
Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz Tao Sun Congliang Chen Peng Qiao Li Shen Xinwang Liu Dongsheng Li 157 6 0 23 Oct 2023
ChannelComp: A General Method for Computation by CommunicationsIEEE Transactions on Communications (IEEE Trans. Commun.), 2023 Saeed Razavikia José Mairton Barros da Silva Carlo Fischione 183 28 0 10 Oct 2023
Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts Lizhang Chen Bo Liu Kaizhao Liang Qian Liu ODL 339 26 0 09 Oct 2023
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR) Guo-qing Jiang Jinlong Liu Zixiang Ding Lin Guo W. Lin AI4CE 153 2 0 24 Sep 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 188 1 0 23 Sep 2023
Distributional Shift-Aware Off-Policy Interval Estimation: A Unified Error Quantification Framework Wenzhuo Zhou Yuhan Li Ruoqing Zhu Annie Qu OffRL 241 7 0 23 Sep 2023
Communication-Efficient Federated Learning via Regularized Sparse Random NetworksIEEE Communications Letters (IEEE Commun. Lett.), 2023 Mohamad Mestoukirdi Omid Esrafilian David Gesbert Qianrui Li N. Gresset FedML 156 1 0 19 Sep 2023
A Theoretical and Empirical Study on the Convergence of Adam with an "Exact" Constant Step Size in Non-Convex Settings Alokendu Mazumder Rishabh Sabharwal Manan Tayal Bhartendu Kumar Punit Rathore 255 0 0 15 Sep 2023
On the Implicit Bias of AdamInternational Conference on Machine Learning (ICML), 2023 M. D. Cattaneo Jason M. Klusowski Boris Shigida 340 22 0 31 Aug 2023
Domain Generalization without Excess Empirical RiskNeural Information Processing Systems (NeurIPS), 2023 Ozan Sener V. Koltun 188 9 0 30 Aug 2023
Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation Leander Weber J. Berend Alexander Binder Thomas Wiegand Wojciech Samek Sebastian Lapuschkin Sebastian Lapuschkin 105 1 0 23 Aug 2023
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks Yequan Zhao Xinling Yu Zhixiong Chen Ziyue Liu Sijia Liu Zheng Zhang PINN 161 13 0 18 Aug 2023
Distributed Extra-gradient with Optimal Complexity and Communication GuaranteesInternational Conference on Learning Representations (ICLR), 2023 Ali Ramezani-Kebrya Kimon Antonakopoulos Igor Krawczuk Justin Deschenaux Volkan Cevher 201 4 0 17 Aug 2023
Stochastic Controlled Averaging for Federated Learning with Communication CompressionInternational Conference on Learning Representations (ICLR), 2023 Xinmeng Huang Ping Li Xiaoyun Li 335 245 0 16 Aug 2023
Private Federated Learning with Dynamic Power Control via Non-Coherent Over-the-Air Computation Anbang Zhang Shuaishuai Guo Shuai Liu 89 2 0 05 Aug 2023
Label Inference Attacks against Node-level Vertical Federated GNNs Marco Arazzi Mauro Conti Stefanos Koffas Marina Krček Antonino Nocera S. Picek Jing Xu FedML AAML 207 1 0 04 Aug 2023
An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine LearningIEEE Signal Processing Magazine (IEEE Signal Process. Mag.), 2023 Yihua Zhang Prashant Khanduri Ioannis C. Tsaknakis Yuguang Yao Min-Fong Hong Sijia Liu AI4CE 293 46 0 01 Aug 2023
AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection StrategyIEEE Transactions on Mobile Computing (IEEE TMC), 2023 Zihao Zhao Yuzhu Mao Zhenpeng Shi Yang Liu Tian-Shing Lan Wenbo Ding Xiaoping Zhang 292 15 0 01 Aug 2023
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive NetworksIEEE Transactions on Mobile Computing (IEEE TMC), 2023 Natalie Lang Stefano Rini Rafael G. L. DÓliveira S. E. Rouayheb FedML 345 8 0 01 Aug 2023
Samplable Anonymous Aggregation for Private Federated Data AnalysisConference on Computer and Communications Security (CCS), 2023 Kunal Talwar Shan Wang Audra McMillan Vojta Jina Vitaly Feldman ... Congzheng Song Karl Tarbe Sebastian Vogt L. Winstrom Shundong Zhou FedML 319 17 0 27 Jul 2023
Communication-Efficient Federated Learning over Capacity-Limited Wireless NetworksIEEE Transactions on Cognitive Communications and Networking (IEEE TCCN), 2023 Jae-Bok Yun Yong-Nam Oh Yo-Seb Jeon H. Vincent Poor 242 3 0 20 Jul 2023
Communication-Efficient Split Learning via Adaptive Feature-Wise CompressionIEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2023 Yong-Nam Oh Jaeho Lee Christopher G. Brinton Yo-Seb Jeon MQ 313 15 0 20 Jul 2023
Accelerating Distributed ML Training via Selective SynchronizationIEEE International Conference on Cluster Computing (CLUSTER), 2023 S. Tyagi Martin Swany FedML 284 7 0 16 Jul 2023
Multiplicative update rules for accelerating deep learning training and increasing robustnessNeurocomputing (Neurocomputing), 2023 Manos Kirtas Nikolaos Passalis Anastasios Tefas AAML OOD 150 7 0 14 Jul 2023
FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive DropoutIEEE International Parallel and Distributed Processing Symposium (IPDPS), 2023 Jingjing Xue Min Liu Sheng Sun Yuwei Wang Hui Jiang Xue Jiang 252 8 0 14 Jul 2023
Over-The-Air Federated Learning: Status Quo, Open Challenges, and Future DirectionsFundamental Research (FR), 2023 Bingnan Xiao Xichen Yu Wei Ni Xin Wang H. Vincent Poor 174 29 0 03 Jul 2023
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles Le‐Yu Chen Yaohua Ma J.N. Zhang 334 9 0 26 Jun 2023
Adaptive Compression in Federated Learning via Side InformationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Berivan Isik Francesco Pase Deniz Gunduz Sanmi Koyejo Tsachy Weissman M. Zorzi FedML 140 16 0 22 Jun 2023
An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning Cheng Yang Xue Yang Dongxian Wu Xiaohu Tang FedML 249 1 0 21 Jun 2023
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant Xianbiao Qi Jianan Wang Lei Zhang 167 0 0 15 Jun 2023
Evaluation and Optimization of Gradient Compression for Distributed Deep LearningIEEE International Conference on Distributed Computing Systems (ICDCS), 2023 Lin Zhang Longteng Zhang Shaoshuai Shi Xiaowen Chu Yue Liu OffRL 134 11 0 15 Jun 2023
GQFedWAvg: Optimization-Based Quantized Federated Learning in General Edge Computing SystemsIEEE Transactions on Wireless Communications (IEEE TWC), 2023 Yangchen Li Ying Cui Vincent K. N. Lau FedML 204 4 0 13 Jun 2023