Towards a Better Theoretical Understanding of Independent Subnetwork Training

28 June 2023

Papers citing "Towards a Better Theoretical Understanding of Independent Subnetwork Training"

14 / 14 papers shown

Title
Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout Chen Dun Mirian Hipolito Garcia C. Jermaine Dimitrios Dimitriadis Anastasios Kyrillidis 52 20 0 28 Oct 2022
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression Kaja Gruntkowska A. Tyurin Peter Richtárik 38 21 0 30 Sep 2022
Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization Soumia Boucherouite Grigory Malinovsky Peter Richtárik El Houcine Bergou 16 3 0 16 Sep 2022
Federated Pruning: Improving Neural Network Efficiency with Federated Learning Rongmei Lin Yonghui Xiao Tien-Ju Yang Ding Zhao Li Xiong Giovanni Motta Franccoise Beaufays FedML 23 12 0 14 Sep 2022
FL_PyTorch: optimization research simulator for federated learning Konstantin Burlachenko Samuel Horváth Peter Richtárik FedML 29 18 0 07 Feb 2022
Optimal Algorithms for Decentralized Stochastic Variational Inequalities D. Kovalev Aleksandr Beznosikov Abdurakhmon Sadiev Michael Persiianov Peter Richtárik Alexander Gasnikov 33 34 0 06 Feb 2022
Permutation Compressors for Provably Faster Distributed Nonconvex Optimization Rafal Szlendak A. Tyurin Peter Richtárik 115 35 0 07 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 44 44 0 07 Oct 2021
Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices Dingzhu Wen Ki-Jun Jeon Kaibin Huang FedML 68 90 0 30 Sep 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 173 411 0 14 Jul 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 176 267 0 26 Feb 2021
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 55 30 0 13 Feb 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019
New Convergence Aspects of Stochastic Gradient Algorithms Lam M. Nguyen Phuong Ha Nguyen Peter Richtárik K. Scheinberg Martin Takáč Marten van Dijk 18 65 0 10 Nov 2018