SGD Learns the Conjugate Kernel Class of the Network

27 February 2017

Papers citing "SGD Learns the Conjugate Kernel Class of the Network"

50 / 130 papers shown

Title
Training NTK to Generalize with KARE Johannes Schwab Bryan Kelly Semyon Malamud Teng Andrea Xu 16 0 0 16 May 2025
The Power of Random Features and the Limits of Distribution-Free Gradient Descent Ari Karchmer Eran Malach 26 0 0 15 May 2025
Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality Ruijia Zhang Siliang Zeng Chenliang Li Alfredo García Mingyi Hong 69 0 0 22 Mar 2025
SHAP values via sparse Fourier representation Ali Gorji Andisheh Amrollahi A. Krause FAtt 38 0 0 08 Oct 2024
Neural Lineage Runpeng Yu Xinchao Wang 40 4 0 17 Jun 2024
Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime A. Shilton Sunil R. Gupta Santu Rana Svetha Venkatesh 34 0 0 24 May 2024
Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks Adeyemi Damilare Adeoye Philipp Christian Petersen Alberto Bemporad 30 1 0 23 Apr 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang YunLong Yu CLL 44 3 0 19 Mar 2024
RedEx: Beyond Fixed Representation Methods via Convex Optimization Amit Daniely Mariano Schain Gilad Yehudai 27 0 0 15 Jan 2024
Weak Correlations as the Underlying Principle for Linearization of Gradient-Based Learning Systems Ori Shem-Ur Yaron Oz 21 0 0 08 Jan 2024
Applying statistical learning theory to deep learning Cédric Gerbelot Avetik G. Karagulyan Stefani Karp Kavya Ravichandran Menachem Stern Nathan Srebro FedML 24 2 0 26 Nov 2023
Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network? Lianke Qin Zhao Song Baocheng Sun 25 7 0 14 Sep 2023
A faster and simpler algorithm for learning shallow networks Sitan Chen Shyam Narayanan 41 7 0 24 Jul 2023
What can a Single Attention Layer Learn? A Study Through the Random Features Lens Hengyu Fu Tianyu Guo Yu Bai Song Mei MLT 40 22 0 21 Jul 2023
Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space Zhengdao Chen 44 1 0 03 Jul 2023
$The $L^\infty$ Learnability of Reproducing Kernel Hilbert Spaces$ The $L^\infty$ Learnability of Reproducing Kernel Hilbert Spaces Hongrui Chen Jihao Long Lei Wu 22 0 0 05 Jun 2023
Most Neural Networks Are Almost Learnable Amit Daniely Nathan Srebro Gal Vardi 31 0 0 25 May 2023
A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networks Ali Gorji Andisheh Amrollahi A. Krause 19 4 0 16 May 2023
Efficient Asynchronize Stochastic Gradient Algorithm with Structured Data Zhao Song Mingquan Ye 32 4 0 13 May 2023
On the average-case complexity of learning output distributions of quantum circuits A. Nietner M. Ioannou R. Sweke R. Kueng Jens Eisert M. Hinsche J. Haferkamp 36 11 0 09 May 2023
Learning Narrow One-Hidden-Layer ReLU Networks Sitan Chen Zehao Dou Surbhi Goel Adam R. Klivans Raghu Meka MLT 27 13 0 20 Apr 2023
Solving Regularized Exp, Cosh and Sinh Regression Problems Zhihang Li Zhao Song Dinesh Manocha 36 39 0 28 Mar 2023
Precise Asymptotic Analysis of Deep Random Feature Models David Bosch Ashkan Panahi B. Hassibi 35 19 0 13 Feb 2023
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems Xiao-Yang Liu Ming Zhu S. Borst A. Elwalid 40 8 0 04 Feb 2023
Gradient Descent in Neural Networks as Sequential Learning in RKBS A. Shilton Sunil R. Gupta Santu Rana Svetha Venkatesh MLT 19 1 0 01 Feb 2023
A Simple Algorithm For Scaling Up Kernel Methods Tengyu Xu Bryan Kelly Semyon Malamud 25 0 0 26 Jan 2023
Infinite-width limit of deep linear neural networks Lénaïc Chizat Maria Colombo Xavier Fernández-Real Alessio Figalli 31 14 0 29 Nov 2022
Vision Transformers provably learn spatial structure Samy Jelassi Michael E. Sander Yuan-Fang Li ViT MLT 34 75 0 13 Oct 2022
Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm Lechao Xiao Jeffrey Pennington 45 10 0 11 Jul 2022
Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling Hongkang Li Ming Wang Sijia Liu Pin-Yu Chen Jinjun Xiong GNN 16 27 0 07 Jul 2022
Optimal Activation Functions for the Random Features Regression Model Jianxin Wang José Bento 39 3 0 31 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 199 128 0 19 May 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 59 23 0 18 May 2022
Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent Yiping Lu Jose H. Blanchet Lexing Ying 38 7 0 15 May 2022
Learning Neural Contextual Bandits Through Perturbed Rewards Yiling Jia Weitong Zhang Dongruo Zhou Quanquan Gu Hongning Wang 13 13 0 24 Jan 2022
Learning Neural Ranking Models Online from Implicit User Feedback Yiling Jia Hongning Wang 16 6 0 17 Jan 2022
Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets Shao-Bo Lin Yao Wang Ding-Xuan Zhou ODL 26 6 0 28 Nov 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 90 99 0 13 Oct 2021
A spectral-based analysis of the separation between two-layer neural networks and linear methods Lei Wu Jihao Long 26 8 0 10 Aug 2021
Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activations Pranjal Awasthi Alex K. Tang Aravindan Vijayaraghavan MLT 26 20 0 21 Jul 2021
Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks Yuqing Li Yaoyu Zhang Chao Ma CML 18 1 0 30 Mar 2021
Recent Advances in Large Margin Learning Yiwen Guo Changshui Zhang AAML AI4CE 36 13 0 25 Mar 2021
Double-descent curves in neural networks: a new perspective using Gaussian processes Ouns El Harzli Bernardo Cuenca Grau Guillermo Valle Pérez A. Louis 22 6 0 14 Feb 2021
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths Quynh N. Nguyen 53 48 0 24 Jan 2021
Implicit Bias of Linear RNNs M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher 20 11 0 19 Jan 2021
Fundamental Tradeoffs in Distributionally Adversarial Training M. Mehrabi Adel Javanmard Ryan A. Rossi Anup B. Rao Tung Mai AAML 20 18 0 15 Jan 2021
Infinitely Wide Tensor Networks as Gaussian Process Erdong Guo D. Draper 19 2 0 07 Jan 2021
Mathematical Models of Overparameterized Neural Networks Cong Fang Hanze Dong Tong Zhang 37 22 0 27 Dec 2020
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 62 357 0 17 Dec 2020
Towards NNGP-guided Neural Architecture Search Daniel S. Park Jaehoon Lee Daiyi Peng Yuan Cao Jascha Narain Sohl-Dickstein BDL 26 33 0 11 Nov 2020