v1v2v3 (latest)

signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu Wang

Kamyar Azizzadenesheli

Anima Anandkumar

FedML

ODL

ArXiv (abs)PDF HTML Github (86★)

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 592 papers shown

Title
A Survey on Over-the-Air ComputationIEEE Communications Surveys and Tutorials (COMST), 2022 Alphan Șahin Rui Yang 403 130 0 20 Oct 2022
Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated LearningConference on Uncertainty in Artificial Intelligence (UAI), 2022 Ruihan Wu Xiangyu Chen Chuan Guo Kilian Q. Weinberger FedML 119 37 0 19 Oct 2022
Inference in conditioned dynamics through causality restorationScientific Reports (Sci Rep), 2022 Alfredo Braunstein Giovanni Catania Luca DallÁsta M. Mariani Anna Paola Muntoni 119 4 0 18 Oct 2022
ScionFL: Efficient and Robust Secure Quantized Aggregation Y. Ben-Itzhak Helen Mollering Benny Pinkas T. Schneider Ajith Suresh Oleksandr Tkachenko S. Vargaftik Christian Weinert Hossein Yalame Avishay Yanai 189 11 0 13 Oct 2022
Over-the-Air Computation Based on Balanced Number Systems for Federated Edge LearningIEEE Transactions on Wireless Communications (TWC), 2022 Alphan Șahin FedML 220 24 0 13 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and OpportunitiesJournal of machine learning research (JMLR), 2022 Brian Bartoldson B. Kailkhura Davis W. Blalock 252 63 0 13 Oct 2022
Invariant Aggregator for Defending against Federated Backdoor AttacksInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2022 Xiaoya Wang Dimitrios Dimitriadis Oluwasanmi Koyejo Shruti Tople FedML 192 2 0 04 Oct 2022
Distributed Non-Convex Optimization with One-Bit Compressors on Heterogeneous Data: Efficient and Resilient Algorithms Ming Xiang Lili Su FedML 150 5 0 03 Oct 2022
Sparse Random Networks for Communication-Efficient Federated LearningInternational Conference on Learning Representations (ICLR), 2022 Berivan Isik Francesco Pase Deniz Gunduz Tsachy Weissman M. Zorzi FedML 183 61 0 30 Sep 2022
Over-the-Air Computation over Balanced Numerals Alphan Șahin Rui Yang 158 12 0 22 Sep 2022
A Demonstration of Over-the-Air Computation for Federated Edge Learning Alphan Șahin 143 8 0 20 Sep 2022
Personalized Federated Learning with Communication Compression El Houcine Bergou Konstantin Burlachenko Aritra Dutta Peter Richtárik FedML 194 10 0 12 Sep 2022
Robustness to Unbounded Smoothness of Generalized SignSGDNeural Information Processing Systems (NeurIPS), 2022 M. Crawshaw Mingrui Liu Francesco Orabona Wei Zhang Zhenxun Zhuang AAML 212 86 0 23 Aug 2022
Joint Privacy Enhancement and Quantization in Federated LearningIEEE Transactions on Signal Processing (IEEE Trans. Signal Process.), 2022 Natalie Lang Elad Sofer Tomer Shaked Stefano Rini FedML 188 64 0 23 Aug 2022
MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust ClusteringProceedings of the ACM on Measurement and Analysis of Computing Systems (POMACS), 2022 Rui Wang Xingkai Wang H. Chen Jérémie Decouchant S. Picek Ziqiang Liu K. Liang 205 2 0 22 Aug 2022
Practical Vertical Federated Learning with Unsupervised Representation LearningIEEE Transactions on Big Data (TBD), 2022 Zhaomin Wu Yue Liu Bingsheng He FedML 149 54 0 13 Aug 2022
FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated LearningInternational Joint Conference on Artificial Intelligence (IJCAI), 2022 Yuanyuan Chen Zichen Chen Pengcheng Wu Han Yu AI4CE 240 20 0 10 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary SurveyJournal of the Franklin Institute (JFI), 2022 Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 260 68 0 02 Aug 2022
CFLIT: Coexisting Federated Learning and Information TransferIEEE Transactions on Wireless Communications (TWC), 2022 Zehong Lin Hang Liu Y. Zhang 180 13 0 26 Jul 2022
Reconciling Security and Communication Efficiency in Federated LearningIEEE Data Engineering Bulletin (DEB), 2022 Karthik Prasad Sayan Ghosh Graham Cormode Ilya Mironov Ashkan Yousefpour Pierre Stock FedML 144 11 0 26 Jul 2022
Adaptive Step-Size Methods for Compressed SGDIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2022 Adarsh M. Subramaniam A. Magesh Venugopal V. Veeravalli 112 1 0 20 Jul 2022
Moment Centralization based Gradient Descent Optimizers for Convolutional Neural Networks Sumanth Sadu S. Dubey S. Sreeja ODL 109 1 0 19 Jul 2022
A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy Kaan Ozkara Antonious M. Girgis Deepesh Data Suhas Diggavi FedML 145 4 0 05 Jul 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion ApproachIEEE Transactions on Communications (IEEE Trans. Commun.), 2022 Naifu Zhang M. Tao Jia Wang Fan Xu 139 15 0 28 Jun 2022
Recognising Affordances in Predicted Futures to Plan with Consideration of Non-canonical Affordance EffectsIEEE Robotics and Automation Letters (RA-L), 2022 S. Arnold Mami Kuroishi Tadashi Adachi Kimitoshi Yamazaki 107 1 0 22 Jun 2022
Shifted Compression Framework: Generalizations and ImprovementsConference on Uncertainty in Artificial Intelligence (UAI), 2022 Egor Shulgin Peter Richtárik 122 6 0 21 Jun 2022
Demystifying the Adversarial Robustness of Random Transformation DefensesInternational Conference on Machine Learning (ICML), 2022 Chawin Sitawarin Zachary Golan-Strieb David Wagner AAML 199 25 0 18 Jun 2022
Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned DataInternational Conference on Machine Learning (ICML), 2022 Timothy Castiglia Anirban Das Maroun Touma S. Patterson FedML 174 61 0 16 Jun 2022
Asynchronous SGD Beats Minibatch SGD Under Arbitrary DelaysNeural Information Processing Systems (NeurIPS), 2022 Konstantin Mishchenko Francis R. Bach Mathieu Even Blake E. Woodworth 200 69 0 15 Jun 2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and StrongerNeural Information Processing Systems (NeurIPS), 2022 Zhiqi Bu Yu Wang Sheng Zha George Karypis 510 94 0 14 Jun 2022
Distributed Adversarial Training to Robustify Deep Neural Networks at ScaleConference on Uncertainty in Artificial Intelligence (UAI), 2022 Gaoyuan Zhang Songtao Lu Yihua Zhang Xiangyi Chen Pin-Yu Chen Quanfu Fan Lee Martie L. Horesh Min-Fong Hong Sijia Liu OOD 206 14 0 13 Jun 2022
Communication-Efficient Federated Learning over MIMO Multiple Access ChannelsIEEE Transactions on Communications (IEEE Trans. Commun.), 2022 Yo-Seb Jeon Mohammad Mohammadi Amiri Namyoon Lee 116 17 0 12 Jun 2022
Gradient Obfuscation Gives a False Sense of Security in Federated LearningUSENIX Security Symposium (USENIX Security), 2022 Kai Yue Richeng Jin Chau-Wai Wong D. Baron H. Dai FedML 197 66 0 08 Jun 2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication CompressionNeural Information Processing Systems (NeurIPS), 2022 Xinmeng Huang Yiming Chen W. Yin Kun Yuan 299 31 0 08 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with GuaranteesNeural Information Processing Systems (NeurIPS), 2022 Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 354 18 0 02 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 280 10 0 31 May 2022
Efficient-Adam: Communication-Efficient Distributed AdamIEEE Transactions on Signal Processing (IEEE Trans. Signal Process.), 2022 Congliang Chen Li Shen Wei Liu Jianfeng Yao 127 25 0 28 May 2022
ByteComp: Revisiting Gradient Compression in Distributed Training Zhuang Wang Yanghua Peng Yibo Zhu T. Ng 168 2 0 28 May 2022
QUIC-FL: Quick Unbiased Compression for Federated Learning Ran Ben-Basat S. Vargaftik Amit Portnoy Gil Einziger Y. Ben-Itzhak Michael Mitzenmacher FedML 297 14 0 26 May 2022
FedAdapter: Efficient Federated Learning for Modern NLP Dongqi Cai Yaozong Wu Shangguang Wang F. Lin Mengwei Xu FedML AI4CE 160 34 0 20 May 2022
Service Delay Minimization for Federated Learning over Mobile DevicesIEEE Journal on Selected Areas in Communications (JSAC), 2022 Rui Chen Dian Shi Xiaoqi Qin Dongjie Liu Miao Pan Shuguang Cui FedML 185 36 0 19 May 2022
On Distributed Adaptive Optimization with Gradient CompressionInternational Conference on Learning Representations (ICLR), 2022 Xiaoyun Li Belhal Karimi Ping Li 149 31 0 11 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 250 11 0 08 May 2022
Communication-Efficient Adaptive Federated LearningInternational Conference on Machine Learning (ICML), 2022 Yujia Wang Lu Lin Jinghui Chen FedML 265 92 0 05 May 2022
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN TrainingConference on Machine Learning and Systems (MLSys), 2022 Han Hu Chenyu Jiang Yuchen Zhong Size Zheng Chuan Wu Yibo Zhu Yanghua Peng Chuanxiong Guo 150 24 0 05 May 2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning Samuel Horváth Maziar Sanjabi Lin Xiao Peter Richtárik Michael G. Rabbat FedML 225 22 0 27 Apr 2022
How to Attain Communication-Efficient DNN Training? Convert, Compress, CorrectIEEE Internet of Things Journal (IEEE IoT J.), 2022 Zhongzhu Chen Eduin E. Hernandez Yu-Chih Huang Stefano Rini MQ 199 0 0 18 Apr 2022
FedVQCS: Federated Learning via Vector Quantized Compressed SensingIEEE Transactions on Wireless Communications (TWC), 2022 Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 212 16 0 16 Apr 2022
Sign Bit is Enough: A Learning Synchronization Framework for Multi-hop All-reduce with Ultimate CompressionDesign Automation Conference (DAC), 2022 Feijie Wu Shiqi He Song Guo Zhihao Qu Yining Qi W. Zhuang Jie Zhang 123 9 0 14 Apr 2022
Distributed Evolution Strategies for Black-box Stochastic OptimizationIEEE Transactions on Parallel and Distributed Systems (TPDS), 2022 Xiaoyu He Zibin Zheng Chuan Chen Yuren Zhou Chuan Luo Qingwei Lin 126 8 0 09 Apr 2022