On the Optimal Memorization Power of ReLU Neural Networks

On the Optimal Memorization Power of ReLU Neural Networks

7 October 2021

Gal Vardi

Papers citing "On the Optimal Memorization Power of ReLU Neural Networks"

12 / 12 papers shown

Title
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding Kevin Xu Issei Sato 37 3 0 02 Oct 2024
On the Complexity of Neural Computation in Superposition Micah Adler Nir Shavit 112 3 0 05 Sep 2024
Empirical Capacity Model for Self-Attention Neural Networks Aki Härmä M. Pietrasik Anna Wilbik 34 1 0 22 Jul 2024
From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport Quentin Bouniot I. Redko Anton Mallasto Charlotte Laclau Karol Arndt Oliver Struckmeier Markus Heinonen Ville Kyrki Samuel Kaski 54 2 0 17 Oct 2023
Minimum width for universal approximation using ReLU networks on compact domain Namjun Kim Chanho Min Sejun Park VLM 27 10 0 19 Sep 2023
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators? T. Kajitsuka Issei Sato 29 16 0 26 Jul 2023
Memorization Capacity of Multi-Head Attention in Transformers Sadegh Mahdavi Renjie Liao Christos Thrampoulidis 24 22 0 03 Jun 2023
Memorization Capacity of Neural Networks with Conditional Computation Erdem Koyuncu 30 4 0 20 Mar 2023
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power Binghui Li Jikai Jin Han Zhong J. Hopcroft Liwei Wang OOD 74 27 0 27 May 2022
$Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete$ Training Fully Connected Neural Networks is $\exists\mathbb{R}$ -Complete Daniel Bertschinger Christoph Hertrich Paul Jungeblut Tillmann Miltzow Simon Weber OffRL 54 30 0 04 Apr 2022
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 33 95 0 25 Jul 2020
Benefits of depth in neural networks Matus Telgarsky 128 602 0 14 Feb 2016