Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group

24 January 2019

Papers citing "Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group"

38 / 38 papers shown

Title
Compact Recurrent Transformer with Persistent Memory Edison Mucllari Z. Daniels David C. Zhang Qiang Ye CLL VLM 49 0 0 02 May 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 137 84 0 21 Feb 2025
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 74 0 0 28 Jan 2025
Graph Unitary Message Passing Haiquan Qiu Yatao Bian Quanming Yao 37 2 0 17 Mar 2024
Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation Haoqi Wang Zhizhong Li Wayne Zhang 20 2 0 03 Aug 2023
Robust low-rank training via approximate orthonormal constraints Dayana Savostianova Emanuele Zangrando Gianluca Ceruti Francesco Tudisco 24 9 0 02 Jun 2023
Convex optimization over a probability simplex James Chok G. Vasil 25 2 0 15 May 2023
Adaptive-saturated RNN: Remember more with less instability Khoi Minh Nguyen-Duy Quang-Cuong Pham B. T. Nguyen ODL 15 1 0 24 Apr 2023
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold Yanhong Fei Xian Wei Yingjie Liu Zhengyu Li Mingsong Chen 28 6 0 16 Feb 2023
Multi-View Independent Component Analysis with Shared and Individual Sources T. Pandeva Patrick Forré CML 15 5 0 05 Oct 2022
Homotopy-based training of NeuralODEs for accurate dynamics discovery Joon-Hyuk Ko Hankyul Koh Nojun Park W. Jhe 46 8 0 04 Oct 2022
Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks Andrea Ceni ODL 23 3 0 03 Oct 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 11 4 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
Path Development Network with Finite-dimensional Lie Group Representation Han Lou Siran Li Hao Ni 16 7 0 02 Apr 2022
Subspace-based Representation and Learning for Phonotactic Spoken Language Recognition Hung-Shin Lee Yu Tsao Shyh-Kang Jeng Hsin-Min Wang 20 9 0 28 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 43 32 0 10 Mar 2022
Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes Yunjiang Jiang Han Zhang Yiming Qiu Yun Xiao Bo Long Wen-Yun Yang 19 6 0 09 Mar 2022
O-ViT: Orthogonal Vision Transformer Yanhong Fei Yingjie Liu Xian Wei Mingsong Chen ViT 13 8 0 28 Jan 2022
Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks Ekkehard Schnoor Arash Behboodi Holger Rauhut 16 13 0 08 Dec 2021
Target Propagation via Regularized Inversion Vincent Roulet Zaïd Harchaoui BDL AAML 19 4 0 02 Dec 2021
Efficiently Modeling Long Sequences with Structured State Spaces Albert Gu Karan Goel Christopher Ré 52 1,665 0 31 Oct 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 25 338 0 18 Oct 2021
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks E. M. Achour Franccois Malgouyres Franck Mamalet 16 20 0 12 Aug 2021
Coordinate descent on the orthogonal group for recurrent neural network training E. Massart V. Abrol 34 10 0 30 Jul 2021
RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks L. Kozachkov Michaela Ennis Jean-Jacques E. Slotine 19 18 0 16 Jun 2021
RotoGrad: Gradient Homogenization in Multitask Learning Adrián Javaloy Isabel Valera 21 86 0 03 Mar 2021
Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies T. Konstantin Rusch Siddhartha Mishra 19 89 0 02 Oct 2020
Sampling using $SU(N)$ gauge equivariant flows D. Boyda G. Kanwar S. Racanière Danilo Jimenez Rezende M. S. Albergo Kyle Cranmer D. Hackett P. Shanahan 25 127 0 12 Aug 2020
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xinyu Wang Yi Ma Jitendra Malik VLM 30 53 0 30 Jun 2020
Recurrent Quantum Neural Networks Johannes Bausch 21 151 0 25 Jun 2020
Lipschitz Recurrent Neural Networks N. Benjamin Erichson Omri Azencot A. Queiruga Liam Hodgkinson Michael W. Mahoney 30 107 0 22 Jun 2020
Concept Whitening for Interpretable Image Recognition Zhi Chen Yijie Bei Cynthia Rudin FAtt 31 314 0 05 Feb 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 24 102 0 04 Feb 2020
Fine-grained Optimization of Deep Neural Networks Mete Ozay ODL 14 1 0 22 May 2019
NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation Anastasis Kratsios Cody B. Hyndman OOD 25 17 0 31 Aug 2018
Optimization on Submanifolds of Convolution Kernels in CNNs Mete Ozay Takayuki Okatani 45 46 0 22 Oct 2016
Learning Unitary Operators with Help From u(n) Stephanie L. Hyland Gunnar Rätsch 91 41 0 17 Jul 2016