v1v2v3 (latest)

signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu Wang

Kamyar Azizzadenesheli

Anima Anandkumar

FedML

ODL

ArXiv (abs)PDF HTML Github (86★)

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 592 papers shown

Title
Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization Afonso de Sá Delgado Neto Maximilian Egger Mayank Bakshi Rawad Bitar FedML AI4CE 97 3 0 20 Jun 2024
The Implicit Bias of Adam on Separable DataNeural Information Processing Systems (NeurIPS), 2024 Chenyang Zhang Difan Zou Yuan Cao AI4CE 202 19 0 15 Jun 2024
H-Fac: Memory-Efficient Optimization with Factorized Hamiltonian DescentInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Son Nguyen Lizhang Chen Bo Liu Qiang Liu 240 6 0 14 Jun 2024
Minimal Communication-Cost Statistical Learning Romain Chor Abdellatif Zaidi Piotr Krasnowski 124 0 0 12 Jun 2024
Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex OptimizationAnnual Conference Computational Learning Theory (COLT), 2024 Devyani Maladkar Ruichen Jiang Aryan Mokhtari 341 6 0 07 Jun 2024
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction Wei Jiang Sifan Yang Wenhao Yang Lijun Zhang 233 9 0 01 Jun 2024
Near Optimal Decentralized Optimization with Compression and Momentum Tracking Rustem Islamov Yuan Gao Sebastian U. Stich 167 0 0 30 May 2024
Leakage-Resilient and Carbon-Neutral Aggregation Featuring the Federated AI-enabled Critical Infrastructure Zehang Deng Ruoxi Sun Minhui Xue Sheng Wen S. Çamtepe Surya Nepal Yang Xiang 175 9 0 24 May 2024
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization Zhe Li Bicheng Ying Zidong Liu Chaosheng Dong Haibo Yang FedML 434 8 0 24 May 2024
Surge Phenomenon in Optimal Learning Rate and Batch Size ScalingNeural Information Processing Systems (NeurIPS), 2024 Shuaipeng Li Penghao Zhao Hailin Zhang Xingwu Sun Hao Wu ... Zheng Fang Jinbao Xue Yangyu Tao Tengjiao Wang Di Wang 192 21 0 23 May 2024
Comparisons Are All You Need for Optimizing Smooth Functions Chenyi Zhang Tongyang Li AAML 203 3 0 19 May 2024
Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance A. Stollenwerk Laurent Jacques FedML 124 0 0 17 May 2024
Advances in Robust Federated Learning: A Survey with Heterogeneity ConsiderationsIEEE Transactions on Big Data (IEEE Trans. Big Data), 2024 Chuan Chen Tianchi Liao Xiaojun Deng Zihou Wu Sheng Huang Zibin Zheng FedML 308 2 0 16 May 2024
Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and PrivacyIEEE Transactions on Machine Learning in Communications and Networking (IEEE TMLCN), 2024 Feng Wang M. C. Gursoy Senem Velipasalar 179 3 0 15 May 2024
Breaching the Bottleneck: Evolutionary Transition from Reward-Driven Learning to Reward-Agnostic Domain-Adapted Learning in Neuromodulated Neural Nets S. Arnold Reiji Suzuki Takaya Arita Kimitoshi Yamazaki 105 0 0 19 Apr 2024
KDk: A Defense Mechanism Against Label Inference Attacks in Vertical Federated Learning Marco Arazzi S. Nicolazzo Antonino Nocera FedML AAML 164 8 0 18 Apr 2024
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks Matteo Tucat Anirbit Mukherjee Procheta Sen Mingfei Sun Omar Rivasplata MLT 188 1 0 12 Apr 2024
Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models Zihan Fang Zheng Lin Zhe Chen Xianhao Chen Yue Gao Yuguang Fang 211 50 0 09 Apr 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 255 19 0 09 Apr 2024
$Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization$ Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization Shuo Xie Zhiyuan Li OffRL 195 37 0 05 Apr 2024
A Survey on Error-Bounded Lossy Compression for Scientific DatasetsACM Computing Surveys (ACM CSUR), 2024 Sheng Di Jinyang Liu Kai Zhao Xin Liang Robert Underwood ... Jon C. Calhoun Guanpeng Li Kazutomo Yoshii Khalid Ayed Alharthi Franck Cappello AI4CE 191 38 0 03 Apr 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 324 11 0 01 Apr 2024
Communication Efficient Distributed Training with Distributed Lion Bo Liu Lemeng Wu Lizhang Chen Kaizhao Liang Jiaxu Zhu Chen Liang Raghuraman Krishnamoorthi Qiang Liu 271 9 0 30 Mar 2024
Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates Natalie Lang Alejandro Cohen Stefano Rini FedML 189 12 0 27 Mar 2024
Compressed Federated Reinforcement Learning with a Generative Model Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 273 4 0 26 Mar 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 147 2 0 25 Mar 2024
Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers Chengxi Li Mikael Skoglund 250 7 0 19 Mar 2024
Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine UnlearningEuropean Conference on Computer Vision (ECCV), 2024 Chongyu Fan Jiancheng Liu Alfred Hero Sijia Liu MU 273 49 0 12 Mar 2024
Data Poisoning Attacks in Gossip LearningInternational Conference on Advanced Information Networking and Applications (AINA), 2024 Alexandre Pham M. Potop-Butucaru Sébastien Tixeuil Serge Fdida 105 3 0 11 Mar 2024
Robust Federated Learning Mitigates Client-side Training Data Distribution Inference Attacks Yichang Xu Ming Yin Minghong Fang Neil Zhenqiang Gong OOD FedML 188 13 0 05 Mar 2024
Federated Learning via Lattice Joint Source-Channel Coding Seyed Mohammad Azimi-Abarghouyi Lav Varshney FedML 144 6 0 01 Mar 2024
FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmission Zeling Zhang Dongqi Cai Yiran Zhang Mengwei Xu Shangguang Wang Ao Zhou 148 8 0 01 Mar 2024
On the Convergence of Federated Learning Algorithms without Data Similarity Ali Beikmohammadi Sarit Khirirat Sindri Magnússon FedML 228 4 0 29 Feb 2024
Pre-training Differentially Private Models with Limited Public Data Zhiqi Bu Xinwei Zhang Mingyi Hong Sheng Zha George Karypis 261 5 0 28 Feb 2024
Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach Bin Wang Jun Fang Hongbin Li Yonina C. Eldar FedML 151 2 0 28 Feb 2024
Why Transformers Need Adam: A Hessian Perspective Yushun Zhang Congliang Chen Tian Ding Ziniu Li Tian Ding Zhimin Luo 264 75 0 26 Feb 2024
Tracking the Median of Gradients with a Stochastic Proximal Point Method Fabian Schaipp Guillaume Garrigos Umut Simsekli Robert M. Gower 248 1 0 20 Feb 2024
Communication-Efficient Distributed Learning with Local Immediate Error Compensation Yifei Cheng Li Shen Linli Xu Xun Qian Shiwei Wu Yiming Zhou Tie Zhang Dacheng Tao Enhong Chen 160 1 0 19 Feb 2024
Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling Arman Adibi Nicolò Dal Fabbro Luca Schenato Sanjeev R. Kulkarni H. Vincent Poor George J. Pappas Hamed Hassani A. Mitra 291 9 0 19 Feb 2024
TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data Richeng Jin Yujie Gu Kai Yue Xiaofan He Zhaoyang Zhang Huaiyu Dai FedML 238 1 0 16 Feb 2024
FedLion: Faster Adaptive Federated Optimization with Fewer Communication Zhiwei Tang Tsung-Hui Chang 248 10 0 15 Feb 2024
Towards Quantifying the Preconditioning Effect of Adam Rudrajit Das Naman Agarwal Sujay Sanghavi Inderjit S. Dhillon 82 8 0 11 Feb 2024
Expediting In-Network Federated Learning by Voting-Based Consensus Model CompressionIEEE Conference on Computer Communications (INFOCOM), 2024 Xiaoxin Su Yipeng Zhou Laizhong Cui Song Guo FedML 109 4 0 06 Feb 2024
Fed-CVLC: Compressing Federated Learning Communications with Variable-Length CodesIEEE Conference on Computer Communications (INFOCOM), 2024 Xiaoxin Su Yipeng Zhou Laizhong Cui John C. S. Lui Jiangchuan Liu FedML 174 2 0 06 Feb 2024
SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding Chanho Park Namyoon Lee FedML AAML 160 4 0 02 Feb 2024
Delving into Decision-based Black-box Attacks on Semantic Segmentation Zhaoyu Chen Zhengyang Shan Jingwen Chang Kaixun Jiang Dingkang Yang Yiting Cheng Wenqiang Zhang AAML 153 0 0 02 Feb 2024
Activations and Gradients Compression for Model-Parallel TrainingDoklady. Mathematics (Dokl. Math.), 2023 Mikhail Rudakov Aleksandr Beznosikov Yaroslav Kholodov Alexander Gasnikov 279 5 0 15 Jan 2024
Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training Yequan Zhao Xianwei Xiao Xinling Yu Ziyue Liu Zhixiong Chen G. Kurczveil R. Beausoleil Zheng Zhang 228 8 0 31 Dec 2023
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image InterpretationIEEE Transactions on Geoscience and Remote Sensing (TGRS), 2023 Weiying Xie Zixuan Wang Haonan Qin Daixun Li Yunsong Li 163 3 0 29 Dec 2023
Federated Hyperdimensional Computing Kazim Ergun Rishikanth Chandrasekaran Tajana Simunic FedML 113 0 0 26 Dec 2023