High-dimensional manifold of solutions in neural networks: insights from statistical physics

20 February 2025

Papers citing "High-dimensional manifold of solutions in neural networks: insights from statistical physics"

30 / 30 papers shown

Title
Exact full-RSB SAT/UNSAT transition in infinitely wide two-layer neural networksSciPost Physics (SciPost Phys.), 2024 B. Annesi Enrico M. Malatesta Francesco Zamponi 215 7 0 09 Oct 2024
The twin peaks of learning neural networks Elizaveta Demyanenko Christoph Feinauer Enrico M. Malatesta Luca Saglietti 189 0 0 23 Jan 2024
The star-shaped space of solutions of the spherical negative perceptron B. Annesi Clarissa Lauditi Carlo Lucibello Enrico M. Malatesta Gabriele Perugini Fabrizio Pittorino Luca Saglietti 126 18 0 18 May 2023
Typical and atypical solutions in non-convex neural networks with discrete and continuous weightsPhysical Review E (PRE), 2023 Carlo Baldassi Enrico M. Malatesta Gabriele Perugini R. Zecchina MQ 221 20 0 26 Apr 2023
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep NetworksInternational Conference on Learning Representations (ICLR), 2022 Xiang Wang Annie Wang Mo Zhou Rong Ge MoMe 451 10 0 03 Oct 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape GeometryInternational Conference on Machine Learning (ICML), 2022 Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 468 28 0 07 Feb 2022
Binary perceptron: efficient algorithms can find solutions in a rare well-connected clusterSymposium on the Theory of Computing (STOC), 2021 Emmanuel Abbe Shuangping Li Allan Sly MQ 165 41 0 04 Nov 2021
Tractability from overparametrization: The example of the negative perceptronProbability theory and related fields (PTRF), 2021 Andrea Montanari Yiqiao Zhong Kangjie Zhou 116 18 0 28 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural NetworksInternational Conference on Learning Representations (ICLR), 2021 R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 477 269 0 12 Oct 2021
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 268 31 0 01 Oct 2021
Unveiling the structure of wide flat minima in neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta Gabriele Perugini R. Zecchina 358 43 0 02 Jul 2021
What can linear interpolation of neural network loss landscapes tell us?International Conference on Machine Learning (ICML), 2021 Tiffany J. Vlaar Jonathan Frankle MoMe 185 30 0 30 Jun 2021
Proof of the Contiguity Conjecture and Lognormal Limit for the Symmetric PerceptronIEEE Annual Symposium on Foundations of Computer Science (FOCS), 2021 Emmanuel Abbe Shuangping Li Allan Sly 236 50 0 25 Feb 2021
Revisiting "Qualitatively Characterizing Neural Network Optimization Problems" Jonathan Frankle 141 26 0 12 Dec 2020
Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixturesJournal of Statistical Mechanics: Theory and Experiment (JSTAT), 2020 Carlo Baldassi Enrico M. Malatesta Matteo Negri R. Zecchina 175 15 0 27 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving GeneralizationInternational Conference on Learning Representations (ICLR), 2020 Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 618 1,625 0 03 Oct 2020
Anomalous diffusion dynamics of learning in deep neural networksNeural Networks (NN), 2020 Guozhang Chen Chengqing Qu P. Gong 191 23 0 22 Sep 2020
Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activationsPhysical Review Letters (PRL), 2019 Carlo Baldassi Enrico M. Malatesta R. Zecchina MLT 360 50 0 17 Jul 2019
Shaping the learning landscape in neural networks around wide flat minimaProceedings of the National Academy of Sciences of the United States of America (PNAS), 2019 Carlo Baldassi Fabrizio Pittorino R. Zecchina MLT 270 90 0 20 May 2019
Comparing Dynamics: Deep Neural Networks versus Glassy Systems Carlo Albert Levent Sagun Mario Geiger S. Spigler Gerard Ben Arous C. Cammarota Yann LeCun Matthieu Wyart Giulio Biroli AI4CE 261 123 0 19 Mar 2018
Essentially No Barriers in Neural Network Energy LandscapeInternational Conference on Machine Learning (ICML), 2018 Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 395 476 0 02 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNsNeural Information Processing Systems (NeurIPS), 2018 T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 428 839 0 27 Feb 2018
Visualizing the Loss Landscape of Neural NetsNeural Information Processing Systems (NeurIPS), 2017 Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 502 2,104 0 28 Dec 2017
Empirical Analysis of the Hessian of Over-Parametrized Neural NetworksInternational Conference on Learning Representations (ICLR), 2017 Levent Sagun Utku Evci V. U. Güney Yann N. Dauphin Léon Bottou 251 440 0 14 Jun 2017
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 235 252 0 22 Nov 2016
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes Carlo Baldassi C. Borgs J. Chayes Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 243 171 0 20 May 2016
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses Carlo Baldassi Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 180 142 0 18 Sep 2015
Qualitatively characterizing neural network optimization problemsInternational Conference on Learning Representations (ICLR), 2014 Ian Goodfellow Oriol Vinyals Andrew M. Saxe ODL 410 550 0 19 Dec 2014
Origin of the computational hardness for learning with binary synapses Haiping Huang Y. Kabashima 139 60 0 08 Aug 2014
Efficient supervised learning in networks with binary synapsesProceedings of the National Academy of Sciences of the United States of America (PNAS), 2007 Carlo Baldassi Alfredo Braunstein Nicolas Brunel R. Zecchina 452 120 0 09 Jul 2007