Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

25 February 2016

Papers citing "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks"

50 / 957 papers shown

Title
Large-Scale Differentially Private BERT Rohan Anil Badih Ghazi Vineet Gupta Ravi Kumar Pasin Manurangsi 33 131 0 03 Aug 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
Connections between Numerical Algorithms for PDEs and Neural Networks Tobias Alt Karl Schrader M. Augustin Pascal Peter Joachim Weickert PINN 21 21 0 30 Jul 2021
Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs Lorenz Kummer Kevin Sidak Tabea Reichmann Wilfried Gansterer MQ 19 5 0 28 Jul 2021
Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey Lorenz Kummer AI4CE 34 0 0 28 Jul 2021
Pruning Ternary Quantization Danyang Liu Xiangshan Chen Jie Fu Chen-li Ma Xue Liu MQ 36 0 0 23 Jul 2021
On the Variance of the Fisher Information for Deep Learning Alexander Soen Ke Sun FedML FAtt 6 15 0 09 Jul 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 32 1,109 0 07 Jul 2021
Adversarial Auto-Encoding for Packet Loss Concealment Santiago Pascual Joan Serra Jordi Pons 24 27 0 07 Jul 2021
Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin Daniel D. Johnson Jonathan Ho Daniel Tarlow Rianne van den Berg DiffM 18 850 0 07 Jul 2021
Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN Rahma Chaabouni Roberto Dessì Eugene Kharitonov 19 20 0 03 Jul 2021
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay E. Lobacheva M. Kodryan Nadezhda Chirkova A. Malinin Dmitry Vetrov 8 25 0 29 Jun 2021
R-Drop: Regularized Dropout for Neural Networks Xiaobo Liang Lijun Wu Juntao Li Yue Wang Qi Meng Tao Qin Wei Chen M. Zhang Tie-Yan Liu 41 424 0 28 Jun 2021
Stabilizing Equilibrium Models by Jacobian Regularization Shaojie Bai V. Koltun J. Zico Kolter 22 57 0 28 Jun 2021
PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs Pu Ren Chengping Rao Yang Liu Jianxun Wang Hao-Lun Sun DiffM AI4CE 37 192 0 26 Jun 2021
Distilling the Knowledge from Conditional Normalizing Flows Dmitry Baranchuk Vladimir Aliev Artem Babenko BDL 28 2 0 24 Jun 2021
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction Peng Wang Lingjie Liu Yuan-Bin Liu Christian Theobalt Taku Komura Wenping Wang 44 1,682 0 20 Jun 2021
Backward Gradient Normalization in Deep Neural Networks Alejandro Cabana Luis F. Lago-Fernández ODL 8 2 0 17 Jun 2021
Cascading Modular Network (CAM-Net) for Multimodal Image Synthesis Shichong Peng Alireza Moazeni Ke Li GAN 16 0 0 16 Jun 2021
A Flow-Based Neural Network for Time Domain Speech Enhancement Martin Strauss B. Edler 15 33 0 16 Jun 2021
Non-Gradient Manifold Neural Network Rui Zhang Ziheng Jiao Hongyuan Zhang Xuelong Li ODL 6 0 0 15 Jun 2021
UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation Won Jang D. Lim Jaesam Yoon Bongwan Kim Juntae Kim 30 125 0 15 Jun 2021
Recomposing the Reinforcement Learning Building Blocks with Hypernetworks Shai Keynan Elad Sarafian Sarit Kraus OffRL 18 28 0 12 Jun 2021
Evaluating Deep Neural Networks for Image Document Enhancement Lucas N. Kirsten R. Piccoli R. Ribani 13 4 0 11 Jun 2021
Catch-A-Waveform: Learning to Generate Audio from a Single Short Example Gal Greshler Tamar Rott Shaham T. Michaeli 16 25 0 11 Jun 2021
CommAID: Visual Analytics for Communication Analysis through Interactive Dynamics Modeling M. T. Fischer Daniel Seebacher R. Sevastjanova Daniel A. Keim Mennatallah El-Assady 17 10 0 11 Jun 2021
Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka 30 35 0 10 Jun 2021
SDGMNet: Statistic-based Dynamic Gradient Modulation for Local Descriptor Learning Jiayi Ma Yuxin Deng 23 4 0 08 Jun 2021
A Lightweight and Gradient-Stable Neural Layer Yueyao Yu Yin Zhang 23 0 0 08 Jun 2021
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence A. Labatie Dominic Masters Zach Eaton-Rosen Carlo Luschi 25 21 0 07 Jun 2021
Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training Dominic Masters A. Labatie Zach Eaton-Rosen Carlo Luschi 19 11 0 07 Jun 2021
Stochastic Whitening Batch Normalization Shengdong Zhang E. Nezhadarya H. Fashandi Jiayi Liu Darin Graham Mohak Shah 8 10 0 03 Jun 2021
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path X. Y. Han V. Papyan D. Donoho AAML 25 136 0 03 Jun 2021
Continual Learning in Deep Networks: an Analysis of the Last Layer Timothée Lesort Thomas George Irina Rish KELM 21 22 0 03 Jun 2021
Towards Deeper Deep Reinforcement Learning with Spectral Normalization Johan Bjorck Carla P. Gomes Kilian Q. Weinberger 19 23 0 02 Jun 2021
NVC-Net: End-to-End Adversarial Voice Conversion Bac Nguyen Cong Fabien Cardinaux AAML 37 41 0 02 Jun 2021
LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond Wenbo Li Kun Zhou Lu Qi Nianjuan Jiang Jiangbo Lu Jiaya Jia 21 166 0 21 May 2021
High-Fidelity and Low-Latency Universal Neural Vocoder based on Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform Modeling Patrick Lumban Tobing T. Toda 24 8 0 20 May 2021
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective Florin Gogianu Tudor Berariu Mihaela Rosca Claudia Clopath L. Buşoniu Razvan Pascanu 16 52 0 11 May 2021
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka D. Kunin 22 26 0 06 May 2021
PD-GAN: Probabilistic Diverse GAN for Image Inpainting Hongyu Liu Ziyu Wan Wei Huang Yibing Song Xintong Han Jing Liao 22 200 0 05 May 2021
Continuous Face Aging via Self-estimated Residual Age Embedding Zeqi Li R. Jiang P. Aarabi CVBM 29 17 0 30 Apr 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,775 0 29 Apr 2021
Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-size Rahul Madhavan Hemant Makwana 11 0 0 15 Apr 2021
Orthogonalizing Convolutional Layers with the Cayley Transform Asher Trockman J. Zico Kolter 21 109 0 14 Apr 2021
FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion Hirokazu Kameoka Kou Tanaka Takuhiro Kaneko 31 21 0 14 Apr 2021
Very Lightweight Photo Retouching Network with Conditional Sequential Modulation Yihao Liu Jingwen He Xiangyu Chen Zhengwen Zhang Hengyuan Zhao Chao Dong Yu Qiao 13 27 0 13 Apr 2021
Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks Louis Bethune Thibaut Boissin M. Serrurier Franck Mamalet Corentin Friedrich Alberto González Sanz 30 21 0 11 Apr 2021
Deep Implicit Statistical Shape Models for 3D Medical Image Delineation Ashwin Raju S. Miao Mei Han Le Lu Junzhou Huang Adam P. Harrison 3DV MedIm 19 33 0 07 Apr 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 40 1,803 0 05 Apr 2021