Failures of Gradient-Based Deep Learning

23 March 2017

Papers citing "Failures of Gradient-Based Deep Learning"

37 / 37 papers shown

Title
Physics of Skill Learning Ziming Liu Yizhou Liu Eric J. Michaud Jeff Gore Max Tegmark 46 1 0 21 Jan 2025
Probing the Latent Hierarchical Structure of Data via Diffusion Models Antonio Sclocchi Alessandro Favero Noam Itzhak Levi M. Wyart DiffM 33 3 0 17 Oct 2024
From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency Kaiyue Wen Huaqing Zhang Hongzhou Lin Jingzhao Zhang MoE LRM 61 2 0 07 Oct 2024
Stochastic Reservoir Computers Peter J. Ehlers H. Nurdin Daniel Soh 34 3 0 20 May 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 94 18 0 04 Mar 2024
Simple and Effective Transfer Learning for Neuro-Symbolic Integration Alessandro Daniele Tommaso Campari Sagar Malhotra Luciano Serafini 27 1 0 21 Feb 2024
Auto-Regressive Next-Token Predictors are Universal Learners Eran Malach LRM 19 36 0 13 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 48 8 0 07 Sep 2023
Machine learning with tree tensor networks, CP rank constraints, and tensor dropout Hao Chen T. Barthel 42 7 0 30 May 2023
Practically Solving LPN in High Noise Regimes Faster Using Neural Networks Haozhe Jiang Kaiyue Wen Yi-Long Chen 30 0 0 14 Mar 2023
Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms Tim Whitaker 27 1 0 12 Feb 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 34 10 0 31 Jan 2023
SML:Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction Wenlong Deng Lang Lang Z. Liu B. Liu 21 0 0 09 Oct 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 30 123 0 18 Jul 2022
Impartial Games: A Challenge for Reinforcement Learning Bei Zhou Søren Riis 26 6 0 25 May 2022
Single Image Super-Resolution Methods: A Survey Bahattin Can Maral SupR 22 12 0 17 Feb 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Tao Luo FaML 27 65 0 19 Jan 2022
Linear algebra with transformers Franccois Charton AIMat 27 56 0 03 Dec 2021
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks Tolga Ergen Mert Pilanci 24 16 0 18 Oct 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 13 14 0 21 Feb 2021
The Connection Between Approximation, Depth Separation and Learnability in Neural Networks Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 21 20 0 31 Jan 2021
Achieving Adversarial Robustness Requires An Active Teacher Chao Ma Lexing Ying 19 1 0 14 Dec 2020
Computational Separation Between Convolutional and Fully-Connected Networks Eran Malach Shai Shalev-Shwartz 16 26 0 03 Oct 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
Predicting Many Properties of a Quantum System from Very Few Measurements Hsin-Yuan Huang R. Kueng J. Preskill 12 1,075 0 18 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 18 76 0 18 Feb 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 48 271 0 03 Feb 2020
Learning Adaptive Regularization for Image Labeling Using Geometric Assignment Ruben Hühnerbein Fabrizio Savarino Stefania Petra Christoph Schnörr 14 11 0 22 Oct 2019
Is Deeper Better only when Shallow is Good? Eran Malach Shai Shalev-Shwartz 20 45 0 08 Mar 2019
Deep Geodesic Learning for Segmentation and Anatomical Landmarking N. Torosdagli D. Liberton P. Verma M. Sincan Janice S. Lee Ulas Bagci 14 107 0 06 Oct 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 26 134 0 20 Jun 2018
Relational inductive biases, deep learning, and graph networks Peter W. Battaglia Jessica B. Hamrick V. Bapst Alvaro Sanchez-Gonzalez V. Zambaldi ... Pushmeet Kohli M. Botvinick Oriol Vinyals Yujia Li Razvan Pascanu AI4CE NAI 94 3,078 0 04 Jun 2018
Barren plateaus in quantum neural network training landscapes Jarrod R. McClean Sergio Boixo V. Smelyanskiy Ryan Babbush Hartmut Neven 13 1,778 0 29 Mar 2018
Deep Learning as a Mixed Convex-Combinatorial Optimization Problem A. Friesen Pedro M. Domingos 18 20 0 31 Oct 2017
Deep Learning is Robust to Massive Label Noise David Rolnick Andreas Veit Serge J. Belongie Nir Shavit NoLa 25 548 0 30 May 2017
Deep Reinforcement Learning: An Overview Yuxi Li OffRL VLM 104 1,502 0 25 Jan 2017
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014