Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks

14 July 2021

Papers citing "Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks"

6 / 6 papers shown

Title
On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width Satoki Ishikawa Ryo Karakida 15 2 0 19 Dec 2023
Jorge: Approximate Preconditioning for GPU-efficient Second-order Optimization Siddharth Singh Zack Sating A. Bhatele ODL 17 0 0 18 Oct 2023
Eva: A General Vectorized Approximation Framework for Second-order Optimization Lin Zhang S. Shi Bo-wen Li 13 1 0 04 Aug 2023
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates Mohammad Mozaffari Sikan Li Zhao Zhang M. Dehnavi 14 3 0 02 Jun 2023
Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning Lin Zhang S. Shi Wei Wang Bo-wen Li 20 10 0 30 Jun 2022
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang S. Shi Wei Wang Bo-wen Li Xiaowen Chu 17 48 0 10 Mar 2020