Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization

27 June 2015

Papers citing "Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization"

50 / 107 papers shown

Title
Cluster-Aware Multi-Round Update for Wireless Federated Learning in Heterogeneous Environments Pengcheng Sun Erwu Liu Wei Ni Kanglei Yu Rui Wang Abbas Jamalipour FedML 31 0 0 06 May 2025
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training Hiroki Naganuma Xinzhi Zhang Man-Chung Yue Ioannis Mitliagkas Philipp A. Witte Russell J. Hewett Yin Tat Lee 65 0 0 25 Apr 2025
Balancing Label Imbalance in Federated Environments Using Only Mixup and Artificially-Labeled Noise Kyle Rui Sang Tahseen Rabbani Furong Huang FedML 41 0 0 20 Sep 2024
Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching Amit Attia Ofir Gaash Tomer Koren 40 0 0 14 Aug 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 67 0 0 27 Jul 2024
Asynchronous Federated Stochastic Optimization for Heterogeneous Objectives Under Arbitrary Delays Charikleia Iakovidou Kibaek Kim FedML 37 2 0 16 May 2024
Convergence Analysis of Decentralized ASGD Mauro Dalle Lucca Tosi Martin Theobald 35 2 0 07 Sep 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 40 0 0 01 Aug 2023
Robust Fully-Asynchronous Methods for Distributed Training over General Architecture Zehan Zhu Ye Tian Yan Huang Jinming Xu Shibo He OOD 32 2 0 21 Jul 2023
Theoretically Principled Federated Learning for Balancing Privacy and Utility Xiaojin Zhang Wenjie Li Kai Chen Shutao Xia Qian Yang FedML 30 9 0 24 May 2023
Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays Adrian Redder Arunselvan Ramaswamy Holger Karl 25 1 0 11 May 2023
Performance and Energy Consumption of Parallel Machine Learning Algorithms Xidong Wu Preston Brazzle Stephen Cahoon 49 0 0 01 May 2023
Considerations on the Theory of Training Models with Differential Privacy Marten van Dijk Phuong Ha Nguyen FedML 38 2 0 08 Mar 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 37 19 0 01 Feb 2023
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 39 4 0 25 Nov 2022
SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication Marco Bornstein Tahseen Rabbani Evana Wang Amrit Singh Bedi Furong Huang FedML 54 18 0 25 Oct 2022
Scaling up Stochastic Gradient Descent for Non-convex Optimisation S. Mohamad H. Alamri A. Bouchachia 50 3 0 06 Oct 2022
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization Feng Zhu Jingjing Zhang Xin Eric Wang 33 3 0 06 Oct 2022
Semi-Synchronous Personalized Federated Learning over Mobile Edge Networks Chaoqun You Daquan Feng Kun Guo Howard H. Yang Tony Q.S. Quek 38 13 0 27 Sep 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 38 1 0 17 Aug 2022
Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology Yan Huang Ying Sun Zehan Zhu Changzhi Yan Jinming Xu FedML 38 15 0 08 Jul 2022
FederatedScope: A Flexible Federated Learning Platform for Heterogeneity Yuexiang Xie Zhen Wang Dawei Gao Daoyuan Chen Liuyi Yao Weirui Kuang Yaliang Li Bolin Ding Jingren Zhou FedML 37 88 0 11 Apr 2022
Distributed Evolution Strategies for Black-box Stochastic Optimization Xiaoyu He Zibin Zheng Chuan Chen Yuren Zhou Chuan Luo Qingwei Lin 29 5 0 09 Apr 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 29 20 0 12 Feb 2022
Multiplayer Performative Prediction: Learning in Decision-Dependent Games Adhyyan Narang Evan Faulkner Dmitriy Drusvyatskiy Maryam Fazel Lillian J. Ratliff 20 42 0 10 Jan 2022
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework Xupeng Miao Hailin Zhang Yining Shi Xiaonan Nie Zhi-Xin Yang Yangyu Tao Bin Cui 24 57 0 14 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 32 11 0 07 Dec 2021
DSAG: A mixed synchronous-asynchronous iterative method for straggler-resilient learning A. Severinson E. Rosnes S. E. Rouayheb Alexandre Graell i Amat 22 2 0 27 Nov 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 21 0 0 01 Oct 2021
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization Kaiwen Zhou Anthony Man-Cho So James Cheng 32 1 0 30 Sep 2021
Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks Chenyuan Feng Heng Yang Deshun Hu Zhiwei Zhao Tony Q.S. Quek Geyong Min 38 74 0 20 Aug 2021
Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions Arsalan Sharifnassab Saber Salehkaleybar S. J. Golestani FedML 11 0 0 19 Aug 2021
Decentralized Federated Learning: Balancing Communication and Computing Costs Wei Liu Li Chen Wenyi Zhang FedML 27 106 0 26 Jul 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 26 13 0 19 Jul 2021
Distributed stochastic optimization with large delays Zhengyuan Zhou P. Mertikopoulos Nicholas Bambos Peter Glynn Yinyu Ye 28 9 0 06 Jul 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 43 290 0 11 Jun 2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Xinran Gu Kaixuan Huang Jingzhao Zhang Longbo Huang FedML 35 96 0 08 Jun 2021
Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX Chengliang Zhang Junzhe Xia Baichen Yang Huancheng Puyang Wei Wang Ruichuan Chen Istemi Ekin Akkus Paarijaat Aditya Feng Yan FedML 53 39 0 04 May 2021
DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation Wei Ping Fan Wu Yunhui Long Luka Rimanic Ce Zhang Bo Li FedML 47 63 0 20 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 23 11 0 16 Mar 2021
Parareal Neural Networks Emulating a Parallel-in-time Algorithm Zhanyu Ma Jiyang Xie Jingyi Yu AI4CE 33 9 0 16 Mar 2021
EventGraD: Event-Triggered Communication in Parallel Machine Learning Soumyadip Ghosh B. Aquino V. Gupta FedML 26 8 0 12 Mar 2021
Consistent Lock-free Parallel Stochastic Gradient Descent for Fast and Stable Convergence Karl Bäckström Ivan Walulya Marina Papatriantafilou P. Tsigas 29 5 0 17 Feb 2021
Anytime Minibatch with Delayed Gradients H. Al-Lawati S. Draper 30 0 0 15 Dec 2020
Integrating Deep Learning in Domain Sciences at Exascale Rick Archibald E. Chow E. DÁzevedo Jack J. Dongarra M. Eisenbach ... Florent Lopez Daniel Nichols S. Tomov Kwai Wong Junqi Yin PINN 23 5 0 23 Nov 2020
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 42 10 0 27 Oct 2020
Async-RED: A Provably Convergent Asynchronous Block Parallel Stochastic Method using Deep Denoising Priors Yu Sun Jiaming Liu Yiran Sun B. Wohlberg Ulugbek S. Kamilov 44 15 0 03 Oct 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 23 28 0 17 Jul 2020
Distributed Training of Deep Learning Models: A Taxonomic Perspective M. Langer Zhen He W. Rahayu Yanbo Xue 30 76 0 08 Jul 2020