Micro-Batch Training with Batch-Channel Normalization and Weight Standardization

25 March 2019

Siyuan Qiao

Papers citing "Micro-Batch Training with Batch-Channel Normalization and Weight Standardization"

38 / 88 papers shown

Title
Urban Sound Classification : striving towards a fair comparison Augustin Arnault Baptiste Hanssens Nicolas Riche 16 8 0 22 Oct 2020
MimicNorm: Weight Mean and Last BN Layer Mimic the Dynamic of Batch Normalization Wen Fei Wenrui Dai Chenglin Li Junni Zou H. Xiong 17 1 0 19 Oct 2020
Representation learning from videos in-the-wild: An object-centric approach Rob Romijnders Aravindh Mahendran Michael Tschannen Josip Djolonga Marvin Ritter N. Houlsby Mario Lucic OCL SSL 31 8 0 06 Oct 2020
Weight and Gradient Centralization in Deep Neural Networks Wolfgang Fuhl Enkelejda Kasneci ODL 13 18 0 02 Oct 2020
Understanding Self-supervised Learning with Dual Deep Networks Yuandong Tian Lantao Yu Xinlei Chen Surya Ganguli SSL 13 78 0 01 Oct 2020
Normalization Techniques in Training DNNs: Methodology, Analysis and Application Lei Huang Jie Qin Yi Zhou Fan Zhu Li Liu Ling Shao AI4CE 10 254 0 27 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 29 79 0 17 Sep 2020
Deep data compression for approximate ultrasonic image formation G. Pilikos L. Horchens K. Batenburg Tristan van Leeuwen F. Lucka 14 8 0 04 Sep 2020
Fast ultrasonic imaging using end-to-end deep learning G. Pilikos L. Horchens K. Batenburg Tristan van Leeuwen F. Lucka MedIm 15 7 0 04 Sep 2020
CRNNs for Urban Sound Tagging with spatiotemporal context Augustin Arnault Nicolas Riche 17 7 0 24 Aug 2020
T-GD: Transferable GAN-generated Images Detection Framework Hyeonseong Jeon Youngoh Bang Junyaup Kim Simon S. Woo 20 45 0 10 Aug 2020
Actor-Action Video Classification CSC 249/449 Spring 2020 Challenge Report Jing Shi Zhiheng Li Haitian Zheng Yihang Xu Tianyou Xiao ... R. Magnotti A. Sexton Jeet Thaker Oscar Su Chenliang Xu 15 1 0 01 Aug 2020
TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning Han Cai Chuang Gan Ligeng Zhu Song Han 19 51 0 22 Jul 2020
A New Look at Ghost Normalization Neofytos Dimitriou Ognjen Arandjelovic 19 8 0 16 Jul 2020
Parametric Instance Classification for Unsupervised Visual Feature Learning Yue Cao Zhenda Xie B. Liu Yutong Lin Zheng-Wei Zhang Han Hu VLM 26 60 0 25 Jun 2020
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift Zachary Nado Shreyas Padhy D. Sculley Alexander DÁmour Balaji Lakshminarayanan Jasper Snoek OOD AI4TS 30 238 0 19 Jun 2020
New Interpretations of Normalization Methods in Deep Learning Jiacheng Sun Xiangyong Cao Hanwen Liang Weiran Huang Zewei Chen Zhenguo Li 18 34 0 16 Jun 2020
Normalized Convolutional Neural Network Dongsuk Kim Geonhee Lee Myungjae Lee S. Kang Dongmin Kim 18 1 0 11 May 2020
Gradient Centralization: A New Optimization Technique for Deep Neural Networks Hongwei Yong Jianqiang Huang Xiansheng Hua Lei Zhang ODL 13 184 0 03 Apr 2020
Pipelined Backpropagation at Scale: Training Large Models without Batches Atli Kosson Vitaliy Chiley Abhinav Venigalla Joel Hestness Urs Koster 33 33 0 25 Mar 2020
Getting to 99% Accuracy in Interactive Segmentation Marco Forte Brian L. Price Scott D. Cohen N. Xu Franccois Pitié 26 37 0 17 Mar 2020
PowerNorm: Rethinking Batch Normalization in Transformers Sheng Shen Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer BDL 16 16 0 17 Mar 2020
$F$ , $B$ , Alpha Matting Marco Forte François Pitié 17 84 0 17 Mar 2020
Cross-Iteration Batch Normalization Zhuliang Yao Yu Cao Shuxin Zheng Gao Huang Stephen Lin 6 85 0 13 Feb 2020
Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization Junjie Yan Ruosi Wan X. Zhang Wei Zhang Yichen Wei Jian-jun Sun 11 38 0 19 Jan 2020
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks Zhaodong Chen Lei Deng Bangyan Wang Guoqi Li Yuan Xie 29 28 0 01 Jan 2020
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 70 1,182 0 24 Dec 2019
Self-Supervised Learning of Video-Induced Visual Invariances Michael Tschannen Josip Djolonga Marvin Ritter Aravindh Mahendran Xiaohua Zhai N. Houlsby Sylvain Gelly Mario Lucic SSL 16 61 0 05 Dec 2019
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection Lewei Yao Hang Xu Wei Zhang Xiaodan Liang Zhenguo Li 16 79 0 22 Nov 2019
Rethinking Normalization and Elimination Singularity in Neural Networks Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille 12 10 0 21 Nov 2019
Understanding the Disharmony between Weight Normalization Family and Weight Decay: $ε-$ shifted $L_2$ Regularizer Li Xiang Chen Shuo Xia Yan Yang Jian 26 2 0 14 Nov 2019
Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks Yuhang Li Xin Dong Wei Wang MQ 23 254 0 28 Sep 2019
Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection Benjin Zhu Zhengkai Jiang Xiangxin Zhou Zeming Li Gang Yu 3DPC 160 484 0 26 Aug 2019
Positional Normalization Boyi Li Felix Wu Kilian Q. Weinberger Serge J. Belongie 22 91 0 09 Jul 2019
MMDetection: Open MMLab Detection Toolbox and Benchmark Kai-xiang Chen Jiaqi Wang Jiangmiao Pang Yuhang Cao Yu Xiong ... Jingdong Wang Jianping Shi Wanli Ouyang Chen Change Loy Dahua Lin VOS 20 2,823 0 17 Jun 2019
Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks Xiang Li Xiaolin Hu Jian Yang 19 193 0 23 May 2019
Countering Noisy Labels By Learning From Auxiliary Clean Labels Tsung Wei Tsai Chongxuan Li Jun Zhu SSL 8 1 0 23 May 2019
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,326 0 05 Nov 2016