Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels

1 March 2021

Papers citing "Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels"

32 / 32 papers shown

Title
Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributions Elisabetta Cornacchia Dan Mikulincer Elchanan Mossel 138 1 0 10 Feb 2025
On the Complexity of Learning Sparse Functions with Statistical and Gradient Queries Nirmit Joshi Theodor Misiakiewicz Nathan Srebro 87 7 0 08 Jul 2024
RedEx: Beyond Fixed Representation Methods via Convex Optimization Amit Daniely Mariano Schain Gilad Yehudai 63 0 0 15 Jan 2024
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 89 8 0 07 Sep 2023
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs Emmanuel Abbe Elisabetta Cornacchia Aryo Lotfi 82 11 0 29 Jun 2023
Transformers learn through gradual rank increase Enric Boix-Adserà Etai Littwin Emmanuel Abbe Samy Bengio J. Susskind 102 37 0 12 Jun 2023
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks Mohammed Nowaz Rabbani Chowdhury Shuai Zhang Ming Wang Sijia Liu Pin-Yu Chen MoE 94 19 0 07 Jun 2023
Tight conditions for when the NTK approximation is valid Enric Boix-Adserà Etai Littwin 84 0 0 22 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 201 15 0 11 May 2023
Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning François Caron Fadhel Ayed Paul Jung Hoileong Lee Juho Lee Hongseok Yang 129 2 0 02 Feb 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 75 11 0 31 Jan 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 97 55 0 30 Jan 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 225 71 0 27 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 379 50 0 29 Sep 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 87 6 0 19 Sep 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 114 133 0 18 Jul 2022
Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels Mohamad Amin Mohamadi Wonho Bae Danica J. Sutherland 77 21 0 25 Jun 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 113 6 0 10 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 85 10 0 08 Jun 2022
Long-Tailed Learning Requires Feature Learning T. Laurent J. V. Brecht Xavier Bresson VLM 69 1 0 29 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 99 129 0 03 May 2022
An initial alignment between neural network and target is needed for gradient descent to learn Emmanuel Abbe Elisabetta Cornacchia Jan Hązła Christopher Marquis 120 16 0 25 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 103 30 0 15 Feb 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 75 0 0 03 Jan 2022
The staircase property: How hierarchical structure can guide deep learning Emmanuel Abbe Enric Boix-Adserà Matthew Brennan Guy Bresler Dheeraj M. Nagaraj 67 56 0 24 Aug 2021
On the Power of Differentiable Learning versus PAC and SQ Learning Emmanuel Abbe Pritish Kamath Eran Malach Colin Sandon Nathan Srebro MLT 125 23 0 09 Aug 2021
What can linearized neural networks actually say about generalization? Guillermo Ortiz-Jiménez Seyed-Mohsen Moosavi-Dezfooli P. Frossard 79 45 0 12 Jun 2021
A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs Gadi Naveh Zohar Ringel SSL MLT 91 33 0 08 Jun 2021
Learning a Single Neuron with Bias Using Gradient Descent Gal Vardi Gilad Yehudai Ohad Shamir MLT 87 17 0 02 Jun 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks Carles Domingo-Enrich A. Bietti Eric Vanden-Eijnden Joan Bruna BDL 57 9 0 15 Apr 2021
Noether: The More Things Change, the More Stay the Same Grzegorz Gluch R. Urbanke 76 18 0 12 Apr 2021
Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks Yuval Belfer Amnon Geifman Meirav Galun Ronen Basri 74 17 0 07 Apr 2021