Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

25 February 2016

Papers citing "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks"

50 / 957 papers shown

Title
On the Weight Dynamics of Deep Normalized Networks Christian H. X. Ali Mehmeti-Göpel Michael Wand 25 1 0 01 Jun 2023
Normalization Enhances Generalization in Visual Reinforcement Learning Lu Li Jiafei Lyu Guozheng Ma Zilin Wang Zhen Yang Xiu Li Zhiheng Li OOD 17 8 0 01 Jun 2023
Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyond Chuan Chen Zhenpeng Wu Yan-Hao Lai Wen-chao Ou Tianchi Liao Zibin Zheng 22 32 0 01 Jun 2023
Self-supervised Vision Transformers for 3D Pose Estimation of Novel Objects S. Thalhammer Jean-Baptiste Weibel Markus Vincze Jose Garcia-Rodriguez ViT 20 9 0 31 May 2023
Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs Yi Sun Xin Xu J. Li Xiaochang Hu Yifei Shi L. Zeng 19 2 0 31 May 2023
Bringing regularized optimal transport to lightspeed: a splitting method adapted for GPUs Jacob Lindbäck Zesen Wang Mikael Johansson OT 35 1 0 29 May 2023
Intelligent gradient amplification for deep neural networks S. Basodi K. Pusuluri Xueli Xiao Yi Pan ODL 16 1 0 29 May 2023
Towards Consistent Video Editing with Text-to-Image Diffusion Models Zicheng Zhang Bonan Li Xuecheng Nie Congying Han Tiande Guo Luoqi Liu DiffM 20 24 0 27 May 2023
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks Atli Kosson Bettina Messmer Martin Jaggi 24 11 0 26 May 2023
Ghost Noise for Regularizing Deep Neural Networks Atli Kosson Dongyang Fan Martin Jaggi 9 1 0 26 May 2023
SING: A Plug-and-Play DNN Learning Technique Adrien Courtois Damien Scieur Jean-Michel Morel Pablo Arias Thomas Eboli 22 0 0 25 May 2023
Adaptive Federated Pruning in Hierarchical Wireless Networks Xiaonan Liu Shiqiang Wang Yansha Deng A. Nallanathan 37 11 0 15 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 28 7 0 09 May 2023
Random Function Descent Felix Benning L. Döring 11 0 0 02 May 2023
Benchmarking Low-Shot Robustness to Natural Distribution Shifts Aaditya K. Singh Kartik Sarangmath Prithvijit Chattopadhyay Judy Hoffman OOD 36 1 0 21 Apr 2023
Factored Neural Representation for Scene Understanding Yu-Shiang Wong Niloy J. Mitra OCL 26 4 0 21 Apr 2023
Magnitude Invariant Parametrizations Improve Hypernetwork Learning Jose Javier Gonzalez Ortiz John Guttag Adrian V. Dalca 26 7 0 15 Apr 2023
Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation Suho Lee SeungWon Seo Jihyo Kim Yejin Lee Sangheum Hwang 24 5 0 03 Apr 2023
C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptation Nazmul Karim Niluthpol Chowdhury Mithun Abhinav Rajvanshi Han-Pang Chiu S. Samarasekera Nazanin Rahnavard TTA 21 56 0 30 Mar 2023
Wave-U-Net Discriminator: Fast and Lightweight Discriminator for Generative Adversarial Network-Based Speech Synthesis Takuhiro Kaneko Hirokazu Kameoka Kou Tanaka Shogo Seki 15 9 0 24 Mar 2023
A Survey of Historical Learning: Learning Models with Learning History Xiang Li Ge Wu Lingfeng Yang Wenzhe Wang Renjie Song Jian Yang MU AI4TS 23 2 0 23 Mar 2023
An Empirical Analysis of the Shift and Scale Parameters in BatchNorm Y. Peerthum Mark Stamp 11 5 0 22 Mar 2023
Convergence of variational Monte Carlo simulation and scale-invariant pre-training Nilin Abrahamsen Zhiyan Ding Gil Goldshlager Lin Lin DRL 25 2 0 21 Mar 2023
Unit Scaling: Out-of-the-Box Low-Precision Training Charlie Blake Douglas Orr Carlo Luschi MQ 22 7 0 20 Mar 2023
TempT: Temporal consistency for Test-time adaptation O. Mutlu Mohammadmahdi Honarmand Saimourya Surabhi Dennis Paul Wall 22 6 0 19 Mar 2023
Configurable EBEN: Extreme Bandwidth Extension Network to enhance body-conducted speech capture Hauret Julien Joubaud Thomas V. Zimpfer Bavu Éric 6 6 0 17 Mar 2023
Making Batch Normalization Great in Federated Deep Learning Jike Zhong Hong-You Chen Wei-Lun Chao FedML 21 9 0 12 Mar 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse Shuangfei Zhai Tatiana Likhomanenko Etai Littwin Dan Busbridge Jason Ramapuram Yizhe Zhang Jiatao Gu J. Susskind AAML 38 64 0 11 Mar 2023
Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation Mattia Litrico Alessio Del Bue Pietro Morerio UQCV 32 59 0 07 Mar 2023
Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning Xiaoqin Zhang Zunjie Xiao Xiao Wu Jiansheng Fang Junyong Shen Yan Hu Jiang-Dong Liu 24 10 0 03 Mar 2023
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 94 167 0 01 Mar 2023
$(α_D,α_G)$ -GANs: Addressing GAN Training Instabilities via Dual Objectives Monica Welfert Kyle Otstot Gowtham R. Kurri Lalitha Sankar 17 5 0 28 Feb 2023
Singular value decomposition based matrix surgery Jehan Ghafuri S. Jassim 9 0 0 22 Feb 2023
Redes Generativas Adversarias (GAN) Fundamentos Teóricos y Aplicaciones J. D. L. Torre GAN 21 1 0 18 Feb 2023
DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data Ling Yu AI4TS 26 27 0 17 Feb 2023
The Expressive Power of Tuning Only the Normalization Layers Angeliki Giannou Shashank Rajput Dimitris Papailiopoulos 22 8 0 15 Feb 2023
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 22 11 0 14 Feb 2023
Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs Stefan Werner Sebastian Peitz 33 9 0 14 Feb 2023
Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancy Cheolhyoung Lee Kyunghyun Cho 15 0 0 08 Feb 2023
Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance Ian Colbert Alessandro Pappalardo Jakoba Petri-Koenig MQ 11 4 0 31 Jan 2023
Implicit Regularization for Group Sparsity Jiangyuan Li THANH VAN NGUYEN C. Hegde Raymond K. W. Wong 32 9 0 29 Jan 2023
Zero-shot causal learning H. Nilforoshan Michael Moor Yusuf Roohani Yining Chen Anja vSurina Michihiro Yasunaga Sara Oblak J. Leskovec CML BDL OffRL 11 11 0 28 Jan 2023
Cross-domain Neural Pitch and Periodicity Estimation Max Morrison Caedon Hsieh Nathan Pruyne Bryan Pardo 18 17 0 28 Jan 2023
Norm-based Generalization Bounds for Compositionally Sparse Neural Networks Tomer Galanti Mengjia Xu Liane Galanti T. Poggio 16 9 0 28 Jan 2023
Conformal inference is (almost) free for neural networks trained with early stopping Zi-Chen Liang Yan Zhou Matteo Sesia BDL 16 10 0 27 Jan 2023
EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets E. Buhmann Gregor Kasieczka Jesse Thaler 3DPC 25 46 0 17 Jan 2023
Pseudo-Inverted Bottleneck Convolution for DARTS Search Space Arash Ahmadian Louis S.P. Liu Yue Fei Konstantinos N. Plataniotis Mahdi S. Hosseini 19 0 0 31 Dec 2022
Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Network Wenjie Li Juncheng Li Guangwei Gao Weihong Deng Jian Yang Guo-Jun Qi Chia-Wen Lin SupR 30 8 0 29 Dec 2022
Hyperspherical Quantization: Toward Smaller and More Accurate Models Dan Liu X. Chen Chen-li Ma Xue Liu MQ 22 3 0 24 Dec 2022
KL Regularized Normalization Framework for Low Resource Tasks Neeraj Kumar Ankur Narang Brejesh Lall 21 1 0 21 Dec 2022