Learning a Single Neuron with Bias Using Gradient Descent

v1v2 (latest)

Learning a Single Neuron with Bias Using Gradient Descent

2 June 2021

Gal Vardi

ArXiv (abs)PDF HTML

Papers citing "Learning a Single Neuron with Bias Using Gradient Descent"

13 / 13 papers shown

Title
Self-Normalized Resets for Plasticity in Continual Learning Vivek F. Farias Adam D. Jozefiak CLL 76 2 0 26 Oct 2024
Disentangle Sample Size and Initialization Effect on Perfect Generalization for Single-Neuron Target Jiajie Zhao Zhiwei Bai Yaoyu Zhang 63 0 0 22 May 2024
Masks, Signs, And Learning Rate Rewinding Advait Gadhikar R. Burkholz 98 10 0 29 Feb 2024
RedEx: Beyond Fixed Representation Methods via Convex Optimization Amit Daniely Mariano Schain Gilad Yehudai 63 0 0 15 Jan 2024
Should Under-parameterized Student Networks Copy or Average Teacher Weights? Berfin Simsek Amire Bendjeddou W. Gerstner Johanni Brea 93 8 0 03 Nov 2023
Max-affine regression via first-order methods Seonho Kim Kiryung Lee 59 3 0 15 Aug 2023
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning Nikhil Ghosh Spencer Frei Wooseok Ha Ting Yu MLT 61 3 0 06 Aug 2023
Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs D. Chistikov Matthias Englert R. Lazic MLT 98 12 0 10 Jun 2023
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 137 6 0 27 Sep 2022
Agnostic Learning of General ReLU Activation Using Gradient Descent Pranjal Awasthi Alex K. Tang Aravindan Vijayaraghavan MLT 60 7 0 04 Aug 2022
Support Vectors and Gradient Dynamics of Single-Neuron ReLU Networks Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 39 0 0 11 Feb 2022
Optimization-Based Separations for Neural Networks Itay Safran Jason D. Lee 377 14 0 04 Dec 2021
Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels Eran Malach Pritish Kamath Emmanuel Abbe Nathan Srebro 82 39 0 01 Mar 2021