Deep equilibrium networks are sensitive to initialization statistics

19 July 2022

Papers citing "Deep equilibrium networks are sensitive to initialization statistics"

6 / 6 papers shown

Title
Towards training digitally-tied analog blocks via hybrid gradient computation Timothy Nest M. Ernoult 44 1 0 05 Sep 2024
Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures Zenan Ling Longbo Li Zhanbo Feng Yixuan Zhang Feng Zhou Robert C. Qiu Zhenyu Liao 32 4 0 05 Feb 2024
Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks Haobo Song Soumajit Majumder Tao R. Lin VLM 18 0 0 16 Jul 2023
iMixer: hierarchical Hopfield network implies an invertible, implicit and iterative MLP-Mixer Toshihiro Ota Masato Taki 24 2 0 25 Apr 2023
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping James Martens Andy Ballard Guillaume Desjardins G. Swirszcz Valentin Dalibard Jascha Narain Sohl-Dickstein S. Schoenholz 83 43 0 05 Oct 2021
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 347 0 14 Jun 2018