Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit

18 July 2022

Papers citing "Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit"

8 / 108 papers shown

Title
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 11 155 0 19 Oct 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 29 56 0 11 Oct 2022
Omnigrok: Grokking Beyond Algorithmic Data Ziming Liu Eric J. Michaud Max Tegmark 54 76 0 03 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 310 48 0 29 Sep 2022
Feature selection with gradient descent on two-layer networks in low-rotation regimes Matus Telgarsky MLT 15 16 0 04 Aug 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 11 6 0 10 Jun 2022
The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon Vimal Thilak Etai Littwin Shuangfei Zhai Omid Saremi Roni Paiss J. Susskind 18 61 0 10 Jun 2022
On the Power of Differentiable Learning versus PAC and SQ Learning Emmanuel Abbe Pritish Kamath Eran Malach Colin Sandon Nathan Srebro MLT 47 22 0 09 Aug 2021