Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks

16 September 2017

Xianglong Liu

Papers citing "Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks"

32 / 32 papers shown

Title
Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities Rahul Parhi Michael Unser 39 5 0 05 Oct 2023
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold Jun Chen Haishan Ye Mengmeng Wang Tianxin Huang Guangwen Dai Ivor W.Tsang Y. Liu 21 10 0 21 Aug 2023
Group Orthogonalization Regularization For Vision Models Adaptation and Robustness Yoav Kurtz Noga Bar Raja Giryes 24 0 0 16 Jun 2023
Auto-Encoding Goodness of Fit A. Palmer Zhiyi Chi Derek Aguiar J. Bi 38 1 0 12 Oct 2022
AdaWCT: Adaptive Whitening and Coloring Style Injection Antoine Dufour Yohan Poirier-Ginter Alexandre Lessard Ryan Smith Michael Lockyer Jean-François Lalonde 26 1 0 01 Aug 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 6 4 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 17 6 0 12 May 2022
Riemannian Hamiltonian methods for min-max optimization on manifolds Andi Han Bamdev Mishra Pratik Jawanpuria Pawan Kumar Junbin Gao 30 16 0 25 Apr 2022
Approximation of Lipschitz Functions using Deep Spline Neural Networks Sebastian Neumayer Alexis Goujon Pakshal Bohra M. Unser 24 15 0 13 Apr 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 35 37 0 12 Mar 2022
Learning Connectivity with Graph Convolutional Networks for Skeleton-based Action Recognition H. Sahbi GNN 23 27 0 06 Dec 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 43 34 0 23 Sep 2021
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks E. M. Achour Franccois Malgouyres Franck Mamalet 16 20 0 12 Aug 2021
Coordinate descent on the orthogonal group for recurrent neural network training E. Massart V. Abrol 29 10 0 30 Jul 2021
R-Drop: Regularized Dropout for Neural Networks Xiaobo Liang Lijun Wu Juntao Li Yue Wang Qi Meng Tao Qin Wei Chen M. Zhang Tie-Yan Liu 41 424 0 28 Jun 2021
Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity Ryan Henderson Djork-Arné Clevert F. Montanari 27 26 0 11 May 2021
Skeleton-based Hand-Gesture Recognition with Lightweight Graph Convolutional Networks H. Sahbi 3DH GNN 18 3 0 09 Apr 2021
Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse YuXiang Liu Jidong Ge Chuanyi Li Jie Gui 15 2 0 22 Mar 2021
Learning with Hyperspherical Uniformity Weiyang Liu Rongmei Lin Zhen Liu Li Xiong Bernhard Schölkopf Adrian Weller 28 35 0 02 Mar 2021
Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy Mengzhu Wang Xiang Zhang L. Lan Wei Wang Huibin Tan Zhigang Luo AI4CE 37 1 0 28 Dec 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip H. S. Torr CLL 33 113 0 22 Oct 2020
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You X. Wang Yi-An Ma Jitendra Malik VLM 24 53 0 30 Jun 2020
Concept Whitening for Interpretable Image Recognition Zhi Chen Yijie Bei Cynthia Rudin FAtt 25 313 0 05 Feb 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 16 99 0 04 Feb 2020
Orthogonal Wasserstein GANs J. Müller Reinhard Klein Michael Weinmann 40 9 0 29 Nov 2019
On Generalization Bounds of a Family of Recurrent Neural Networks Minshuo Chen Xingguo Li T. Zhao 11 70 0 28 Oct 2019
Attentive Normalization Xilai Li Wei Sun Tianfu Wu OOD ViT 25 31 0 04 Aug 2019
ABD-Net: Attentive but Diverse Person Re-Identification Tianlong Chen Shaojin Ding Jingyi Xie Ye Yuan Wuyang Chen Yang Yang Zhou Ren Zhangyang Wang 27 479 0 03 Aug 2019
Deep Asymmetric Networks with a Set of Node-wise Variant Activation Functions Jinhyeok Jang Hyunjoong Cho Jaehong Kim Jaeyeon Lee Seungjoon Yang 15 2 0 11 Sep 2018
On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond Xingguo Li Junwei Lu Zhaoran Wang Jarvis D. Haupt T. Zhao 25 78 0 13 Jun 2018
geomstats: a Python Package for Riemannian Geometry in Machine Learning Nina Miolane Johan Mathe Claire Donnat Mikael Jorda Xavier Pennec AI4CE 28 123 0 21 May 2018
Optimization on Submanifolds of Convolution Kernels in CNNs Mete Ozay Takayuki Okatani 43 46 0 22 Oct 2016