v1v2v3v4v5v6 (latest)

Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations

17 July 2019

Papers citing "Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations"

23 / 23 papers shown

Title
Statistical mechanics of extensive-width Bayesian neural networks near interpolation Jean Barbier Francesco Camilli Minh-Toan Nguyen Mauro Pastore Rudy Skerk 87 1 0 30 May 2025
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime Francesco Camilli D. Tieplova Eleonora Bergamin Jean Barbier 528 3 0 06 May 2025
High-dimensional manifold of solutions in neural networks: insights from statistical physics Enrico M. Malatesta 194 4 0 20 Feb 2025
Injectivity capacity of ReLU gates Mihailo Stojnic 95 3 0 28 Oct 2024
Exact full-RSB SAT/UNSAT transition in infinitely wide two-layer neural networks B. Annesi Enrico M. Malatesta Francesco Zamponi 131 4 0 09 Oct 2024
Solution space and storage capacity of fully connected two-layer neural networks with generic activation functions Sota Nishiyama Masayuki Ohzeki 117 2 0 20 Apr 2024
Exact objectives of random linear programs and mean widths of random polyhedrons M. Stojnic 49 0 0 06 Mar 2024
Fixed width treelike neural networks capacity analysis -- generic activations M. Stojnic 94 3 0 08 Feb 2024
$\emph{Lifted} RDT based capacity analysis of the 1-hidden layer treelike \emph{sign} perceptrons neural networks$ \emph{Lifted} RDT based capacity analysis of the 1-hidden layer treelike \emph{sign} perceptrons neural networks M. Stojnic 95 1 0 13 Dec 2023
Capacity of the treelike sign perceptrons neural networks with one hidden layer -- RDT based upper bounds M. Stojnic 107 4 0 13 Dec 2023
Fundamental limits of overparametrized shallow neural networks for supervised learning Francesco Camilli D. Tieplova Jean Barbier 93 10 0 11 Jul 2023
Typical and atypical solutions in non-convex neural networks with discrete and continuous weights Carlo Baldassi Enrico M. Malatesta Gabriele Perugini R. Zecchina MQ 132 16 0 26 Apr 2023
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 344 27 0 07 Feb 2022
On neural network kernels and the storage capacity problem Jacob A. Zavatone-Veth Cengiz Pehlevan 77 7 0 12 Jan 2022
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 164 29 0 01 Oct 2021
Entropic alternatives to initialization Daniele Musso 118 1 0 16 Jul 2021
Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application Fabio Amadio Alberto Dalla Libera R. Antonello D. Nikovski R. Carli Diego Romeres 96 32 0 28 Jan 2021
Activation function dependence of the storage capacity of treelike neural networks Jacob A. Zavatone-Veth Cengiz Pehlevan 85 16 0 21 Jul 2020
Partial local entropy and anisotropy in deep weight spaces Daniele Musso 127 3 0 17 Jul 2020
Entropic gradient descent algorithms and wide flat minima Fabrizio Pittorino Carlo Lucibello Christoph Feinauer Gabriele Perugini Carlo Baldassi Elizaveta Demyanenko R. Zecchina ODL MLT 225 35 0 14 Jun 2020
Clustering of solutions in the symmetric binary perceptron Carlo Baldassi R. D. Vecchia Carlo Lucibello R. Zecchina 146 23 0 15 Nov 2019
Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation Elisa Oostwal Michiel Straat Michael Biehl MLT 124 61 0 16 Oct 2019
Shaping the learning landscape in neural networks around wide flat minima Carlo Baldassi Fabrizio Pittorino R. Zecchina MLT 161 87 0 20 May 2019