signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu-Xiang Wang

Kamyar Azizzadenesheli

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 157 papers shown

Title
CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergence Kun-Yen Huang Shin-Yi Pu 30 9 0 14 Jan 2023
A Comprehensive Survey of Dataset Distillation Shiye Lei Dacheng Tao DD 31 87 0 13 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 19 4 0 06 Jan 2023
Efficient On-device Training via Gradient Filtering Yuedong Yang Guihong Li R. Marculescu 31 18 0 01 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning Mingyuan Fan Cen Chen Chengyu Wang Ximeng Liu Wenmeng Zhou Jun Huang AAML FedML 32 0 0 05 Dec 2022
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 14 6 0 31 Oct 2022
An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization Elvin Lo Pin-Yu Chen 24 0 0 27 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 45 8 0 26 Oct 2022
Over-the-Air Computation over Balanced Numerals Alphan Șahin Rui Yang 23 9 0 22 Sep 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 27 46 0 23 Aug 2022
MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering Rui Wang Xingkai Wang H. Chen Jérémie Decouchant S. Picek Z. Liu K. Liang 29 1 0 22 Aug 2022
Practical Vertical Federated Learning with Unsupervised Representation Learning Zhaomin Wu Q. Li Bingsheng He FedML 30 36 0 13 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 49 59 0 02 Aug 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach Naifu Zhang M. Tao Jia Wang Fan Xu 11 13 0 28 Jun 2022
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale Gaoyuan Zhang Songtao Lu Yihua Zhang Xiangyi Chen Pin-Yu Chen Quanfu Fan Lee Martie L. Horesh Min-Fong Hong Sijia Liu OOD 22 12 0 13 Jun 2022
Gradient Obfuscation Gives a False Sense of Security in Federated Learning Kai Yue Richeng Jin Chau-Wai Wong D. Baron H. Dai FedML 26 46 0 08 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 18 9 0 31 May 2022
Efficient-Adam: Communication-Efficient Distributed Adam Congliang Chen Li Shen Wei Liu Z. Luo 23 19 0 28 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 16 10 0 08 May 2022
Communication-Efficient Adaptive Federated Learning Yujia Wang Lu Lin Jinghui Chen FedML 19 69 0 05 May 2022
Enabling All In-Edge Deep Learning: A Literature Review Praveen Joshi Mohammed Hasanuzzaman Chandra Thapa Haithem Afli T. Scully 21 22 0 07 Apr 2022
Nonlinear gradient mappings and stochastic optimization: A general framework with applications to heavy-tail noise D. Jakovetić Dragana Bajović Anit Kumar Sahu S. Kar Nemanja Milošević Dusan Stamenkovic 17 12 0 06 Apr 2022
FedSynth: Gradient Compression via Synthetic Data in Federated Learning Shengyuan Hu Jack Goetz Kshitiz Malik Hongyuan Zhan Zhe Liu Yue Liu DD FedML 24 38 0 04 Apr 2022
Over-the-Air Federated Learning via Second-Order Optimization Peng Yang Yuning Jiang Ting Wang Yong Zhou Yuanming Shi Colin N. Jones 40 28 0 29 Mar 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 22 20 0 12 Feb 2022
SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries J. Akoun S. Meyer AAML FedML 14 1 0 04 Feb 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 15 8 0 09 Jan 2022
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats Brian Chmiel Ron Banner Elad Hoffer Hilla Ben Yaacov Daniel Soudry MQ 25 22 0 19 Dec 2021
Federated Two-stage Learning with Sign-based Voting Zichen Ma Zihan Lu Yu Lu Wenye Li Jinfeng Yi Shuguang Cui FedML 28 2 0 10 Dec 2021
Improving Differentially Private SGD via Randomly Sparsified Gradients Junyi Zhu Matthew B. Blaschko 21 5 0 01 Dec 2021
Communication-Efficient Federated Learning via Quantized Compressed Sensing Yong-Nam Oh Namyoon Lee Yo-Seb Jeon H. Vincent Poor FedML MQ 22 34 0 30 Nov 2021
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 32 416 0 24 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 22 14 0 01 Nov 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 17 23 0 18 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 28 46 0 11 Oct 2021
Federated Learning via Plurality Vote Kai Yue Richeng Jin Chau-Wai Wong H. Dai FedML 21 8 0 06 Oct 2021
Bayesian AirComp with Sign-Alignment Precoding for Wireless Federated Learning Chanhoo Park Seunghoon Lee Namyoon Lee 29 5 0 14 Sep 2021
Fundamental limits of over-the-air optimization: Are analog schemes optimal? Shubham K. Jha Prathamesh Mayekar Himanshu Tyagi 22 7 0 11 Sep 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 39 37 0 25 Aug 2021
A Hard Label Black-box Adversarial Attack Against Graph Neural Networks Jiaming Mu Binghui Wang Qi Li Kun Sun Mingwei Xu Zhuotao Liu AAML 15 33 0 21 Aug 2021
Decentralized Composite Optimization with Compression Yao Li Xiaorui Liu Jiliang Tang Ming Yan Kun Yuan 19 9 0 10 Aug 2021
Evaluating Federated Learning for Intrusion Detection in Internet of Things: Review and Challenges Enrique Mármol Campos Pablo Fernández Saura Aurora González-Vidal José Luis Hernández Ramos Jorge Bernal Bernabé G. Baldini A. Gómez-Skarmeta 31 149 0 02 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 173 411 0 14 Jul 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 23 30 0 03 Jul 2021
LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update Jiawei Zhao Steve Dai Rangharajan Venkatesan Brian Zimmer Mustafa Ali Ming-Yu Liu Brucek Khailany B. Dally Anima Anandkumar MQ 23 13 0 26 Jun 2021
DP-SIGNSGD: When Efficiency Meets Privacy and Robustness Lingjuan Lyu FedML AAML 14 19 0 11 May 2021
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates Laizhong Cui Xiaoxin Su Yipeng Zhou Yi Pan FedML 22 35 0 10 May 2021
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges Mingzhe Chen Deniz Gündüz Kaibin Huang Walid Saad M. Bennis Aneta Vulgarakis Feljan H. Vincent Poor 21 401 0 05 Apr 2021