Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?

22 October 2018

ArXiv (abs)PDF HTML Github (129★)

Papers citing "Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?"

50 / 107 papers shown

Title
Quantum-PEFT: Ultra parameter-efficient fine-tuning Toshiaki Koike-Akino F. Tonin Yongtao Wu Frank Zhengqing Wu Leyla Naz Candogan Volkan Cevher MQ 219 5 0 07 Mar 2025
Lightweight Frequency Masker for Cross-Domain Few-Shot Semantic Segmentation Jintao Tong Yixiong Zou Yuhua Li Ruixuan Li 96 6 0 29 Oct 2024
Learning in Wilson-Cowan model for metapopulation Raffaele Marino L. Buffoni Lorenzo Chicchi F. Patti Diego Febbe Lorenzo Giambagli Duccio Fanelli 70 2 0 24 Jun 2024
FORML: A Riemannian Hessian-free Method for Meta-learning on Stiefel Manifolds H. Tabealhojeh S. Roy Peyman Adibi Hossein Karshenas 79 0 0 28 Feb 2024
Towards Better Orthogonality Regularization with Disentangled Norm in Training Deep CNNs Changhao Wu Shenan Zhang Fangsong Long Ziliang Yin Tuo Leng 20 1 0 16 Jun 2023
Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers F. Tonin Panagiotis Patrinos Johan A. K. Suykens 41 0 0 12 Jun 2023
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold S. Schechtman D. Tiapkin Michael Muehlebach Eric Moulines 75 8 0 16 Mar 2023
Distortion-Disentangled Contrastive Learning Jinfeng Wang Sifan Song Jionglong Su S. Kevin Zhou SSL 104 5 0 09 Mar 2023
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold Yanhong Fei Xian Wei Yingjie Liu Zhengyu Li Mingsong Chen 95 6 0 16 Feb 2023
Look Beyond Bias with Entropic Adversarial Data Augmentation Thomas Duboudin Emmanuel Dellandrea Corentin Abgrall Gilles Hénaff Liming Chen CML 69 4 0 10 Jan 2023
Orthogonal SVD Covariance Conditioning and Latent Disentanglement Yue Song N. Sebe Wei Wang 79 6 0 11 Dec 2022
TAOTF: A Two-stage Approximately Orthogonal Training Framework in Deep Neural Networks Taoyong Cui Jianze Li Yuhan Dong Li Liu 47 1 0 25 Nov 2022
On the biological plausibility of orthogonal initialisation for solving gradient instability in deep neural networks Nikolay Manchev Michael W. Spratling ODL 19 1 0 27 Oct 2022
Discriminatory and orthogonal feature learning for noise robust keyword spotting Donghyeon Kim Kyungdeuk Ko D. Han Hanseok Ko 52 3 0 20 Oct 2022
Batch Normalization Explained Randall Balestriero Richard G. Baraniuk AAML 92 17 0 29 Sep 2022
Improving GANs for Long-Tailed Data through Group Spectral Regularization Harsh Rangwani Naman Jaswani Tejan Karmali Varun Jampani R. Venkatesh Babu 53 21 0 21 Aug 2022
What can we Learn by Predicting Accuracy? Olivier Risser-Maroix Benjamin Chamand 62 4 0 02 Aug 2022
Riemannian Stochastic Gradient Method for Nested Composition Optimization Dewei Zhang S. Tajbakhsh 89 1 0 19 Jul 2022
Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality Yue Song N. Sebe Wei Wang 88 8 0 05 Jul 2022
Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search Taehyeon Kim Heesoo Myeong Se-Young Yun 74 2 0 27 Jun 2022
Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth Nitin Bansal Pan Ji Junsong Yuan Yi Tian Xu MDE 104 4 0 21 Jun 2022
Maximum Class Separation as Inductive Bias in One Matrix Tejaswi Kasarla Gertjan J. Burghouts Max van Spengler Elise van der Pol Rita Cucchiara Pascal Mettes 72 22 0 17 Jun 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 78 5 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 58 6 0 12 May 2022
Riemannian Hamiltonian methods for min-max optimization on manifolds Andi Han Bamdev Mishra Pratik Jawanpuria Pawan Kumar Junbin Gao 80 17 0 25 Apr 2022
Learning Decoupling Features Through Orthogonality Regularization Li Wang Rongzhi Gu Weiji Zhuang Peng Gao Yujun Wang Yuexian Zou 46 1 0 31 Mar 2022
CHEX: CHannel EXploration for CNN Model Compression Zejiang Hou Minghai Qin Fei Sun Xiaolong Ma Kun Yuan Yi Xu Yen-kuang Chen Rong Jin Yuan Xie S. Kung 79 74 0 29 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 109 42 0 12 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 102 32 0 10 Mar 2022
Orthogonalising gradients to speed up neural network optimisation Mark Tuddenham Adam Prugel-Bennett Jonathan Hare ODL 42 5 0 14 Feb 2022
Deep Learning meets Liveness Detection: Recent Advancements and Challenges Arian Sabaghi Marzieh Oghbaie Kooshan Hashemifard Mohammad Akbari AAML 63 7 0 29 Dec 2021
Cross-domain User Preference Learning for Cold-start Recommendation Huiling Zhou Jie Liu Zhikang Li Jin Yu Hongxia Yang 49 0 0 07 Dec 2021
Clustering Effect of (Linearized) Adversarial Robust Models Yang Bai Xin Yan Yong Jiang Shutao Xia Yisen Wang OOD AAML 74 5 0 25 Nov 2021
Exploiting a Zoo of Checkpoints for Unseen Tasks Jiaji Huang Qiang Qiu Kenneth Church 67 4 0 05 Nov 2021
Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning Uche M. Osahor Nasser M. Nasrabadi 61 11 0 18 Oct 2021
Nonlinear Transform Induced Tensor Nuclear Norm for Tensor Completion Ben-Zheng Li Xile Zhao Teng-Yu Ji Xiongjun Zhang Tingzhu Huang MedIm 78 47 0 17 Oct 2021
Beyond Neighbourhood-Preserving Transformations for Quantization-Based Unsupervised Hashing S. Hemati H. R. Tizhoosh MQ 18 2 0 01 Oct 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Helen Zhou Yu Li Yi Chang Xin Wang 99 34 0 23 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks K. Aoyama H. Sawada 52 1 0 01 Sep 2021
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks El Mehdi Achour Franccois Malgouyres Franck Mamalet 62 21 0 12 Aug 2021
Coordinate descent on the orthogonal group for recurrent neural network training E. Massart V. Abrol 66 11 0 30 Jul 2021
Better Training using Weight-Constrained Stochastic Dynamics Benedict Leimkuhler Tiffany J. Vlaar Timothée Pouchon Amos Storkey 43 9 0 20 Jun 2021
Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for Better Single-Source Domain Generalization Thomas Duboudin Emmanuel Dellandrea Corentin Abgrall Gilles Hénaff Liming Chen 48 9 0 15 Jun 2021
Taxonomy of Machine Learning Safety: A Survey and Primer Sina Mohseni Haotao Wang Zhiding Yu Chaowei Xiao Zhangyang Wang J. Yadawa 84 32 0 09 Jun 2021
An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks Cong Xu Xiang Li Min Yang AAML 58 15 0 19 May 2021
Dual Metric Learning for Effective and Efficient Cross-Domain Recommendations Pan Li Alexander Tuzhilin 59 53 0 17 Apr 2021
Uncertainty Surrogates for Deep Learning R. Achanta Natasa Tagasovska OOD UQCV 48 0 0 16 Apr 2021
Orthogonalizing Convolutional Layers with the Cayley Transform Asher Trockman J. Zico Kolter 95 115 0 14 Apr 2021
Orthogonal Projection Loss Kanchana Ranasinghe Muzammal Naseer Munawar Hayat Salman Khan Fahad Shahbaz Khan VLM 67 73 0 25 Mar 2021
Preprint: Norm Loss: An efficient yet effective regularization method for deep neural networks Theodoros Georgiou Sebastian Schmitt Thomas Bäck Wei Chen M. Lew ODL 34 3 0 11 Mar 2021