Parallel training of DNNs with Natural Gradient and Parameter Averaging

27 October 2014

Sanjeev Khudanpur

Papers citing "Parallel training of DNNs with Natural Gradient and Parameter Averaging"

28 / 28 papers shown

Title
Using Kaldi for Automatic Speech Recognition of Conversational Austrian German J. Linke Saskia Wepner G. Kubin Barbara Schuppler 22 4 0 16 Jan 2023
End-to-End Spoken Language Understanding: Performance analyses of a voice command task in a low resource setting Thierry Desot François Portet Michel Vacher 27 12 0 17 Jul 2022
Certified Robustness in Federated Learning Motasem Alfarra Juan C. Pérez Egor Shulgin Peter Richtárik Guohao Li AAML FedML 23 7 0 06 Jun 2022
Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Model Yingli Shen Huiyu Yang Lin Lin 19 96 0 15 Feb 2022
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 43 19 0 15 Oct 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
Comparing CTC and LFMMI for out-of-domain adaptation of wav2vec 2.0 acoustic model Apoorv Vyas S. Madikeri H. Bourlard 16 15 0 06 Apr 2021
Pkwrap: a PyTorch Package for LF-MMI Training of Acoustic Models S. Madikeri Sibo Tong Juan Pablo Zuluaga Apoorv Vyas P. Motlícek H. Bourlard VLM 6 19 0 07 Oct 2020
PSO-PS: Parameter Synchronization with Particle Swarm Optimization for Distributed Training of Deep Neural Networks Qing Ye Y. Han Yanan Sun Jiancheng Lv 28 3 0 06 Sep 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
Federated learning with hierarchical clustering of local updates to improve training on non-IID data Christopher Briggs Zhong Fan Péter András FedML 20 556 0 24 Apr 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 27 152 0 30 Dec 2019
Variational Federated Multi-Task Learning Luca Corinzia Ami Beuret J. M. Buhmann FedML 27 159 0 14 Jun 2019
Collaborative Deep Learning Across Multiple Data Centers Kele Xu Haibo Mi Dawei Feng Huaimin Wang Chuan Chen Zibin Zheng Xu Lan FedML 81 17 0 16 Oct 2018
Federated Learning for Keyword Spotting David Leroy A. Coucke Thibaut Lavril Thibault Gisselbrecht Joseph Dureau FedML 13 282 0 09 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 33 348 0 22 Aug 2018
cpSGD: Communication-efficient and differentially-private distributed SGD Naman Agarwal A. Suresh Felix X. Yu Sanjiv Kumar H. B. McMahan FedML 28 485 0 27 May 2018
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning Tribhuvanesh Orekondy Seong Joon Oh Yang Zhang Bernt Schiele Mario Fritz PICV FedML 359 37 0 15 May 2018
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 33 702 0 26 Feb 2018
Using of heterogeneous corpora for training of an ASR system J. Trmal Manish Kumar Vimal Manohar Sanjeev Khudanpur Matt Post Paul McNamee 18 1 0 01 Jun 2017
Deep Speaker Feature Learning for Text-independent Speaker Verification Lantian Li Yixiang Chen Ying Shi Zhiyuan Tang Dong Wang 13 86 0 10 May 2017
Distributed Mean Estimation with Limited Communication A. Suresh Felix X. Yu Sanjiv Kumar H. B. McMahan FedML 22 360 0 02 Nov 2016
Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition Zhiyuan Tang Ying Shi Dong Wang Yang Feng Shiyue Zhang 22 53 0 28 Sep 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 26 17,032 0 17 Feb 2016
A Kronecker-factored approximate Fisher matrix for convolution layers Roger C. Grosse James Martens ODL 29 258 0 03 Feb 2016
Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications to Parallel Machine Learning and Multi-Label Image Segmentation Kai Wei Rishabh K. Iyer Shengjie Wang Wenruo Bai J. Bilmes 19 37 0 29 Oct 2015
Experiments on Parallel Training of Deep Neural Network using Model Averaging Hang Su Haoyu Chen MoMe FedML 21 70 0 05 Jul 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 46 43,015 0 11 Feb 2015