Understanding deep learning requires rethinking generalization

10 November 2016

Benjamin Recht

Papers citing "Understanding deep learning requires rethinking generalization"

50 / 1,235 papers shown

Title
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates Ning Liu Xiaolong Ma Zhiyuan Xu Yanzhi Wang Jian Tang Jieping Ye 43 185 0 06 Jul 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 49 55 0 05 Jul 2019
Invariant Risk Minimization Martín Arjovsky Léon Bottou Ishaan Gulrajani David Lopez-Paz OOD 116 2,177 0 05 Jul 2019
Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation Shuo-feng Zhang Lei Xie GNN 34 54 0 04 Jul 2019
Deep Gamblers: Learning to Abstain with Portfolio Theory Liu Ziyin Zhikang T. Wang Paul Pu Liang Ruslan Salakhutdinov Louis-Philippe Morency Masahito Ueda 40 110 0 29 Jun 2019
High-Dimensional Optimization in Adaptive Random Subspaces Jonathan Lacotte Mert Pilanci Marco Pavone 33 16 0 27 Jun 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 24 766 0 26 Jun 2019
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness Fanny Yang Zuowen Wang C. Heinze-Deml 33 42 0 26 Jun 2019
Further advantages of data augmentation on convolutional neural networks Alex Hernández-García Peter König 22 107 0 26 Jun 2019
Importance Estimation for Neural Network Pruning Pavlo Molchanov Arun Mallya Stephen Tyree I. Frosio Jan Kautz 3DPC 42 862 0 25 Jun 2019
On the Noisy Gradient Descent that Generalizes as SGD Jingfeng Wu Wenqing Hu Haoyi Xiong Jun Huan Vladimir Braverman Zhanxing Zhu MLT 29 10 0 18 Jun 2019
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias Stéphane dÁscoli Levent Sagun Joan Bruna Giulio Biroli 27 36 0 16 Jun 2019
Empirical study of extreme overfitting points of neural networks D. Merkulov Ivan Oseledets 3DPC 24 7 0 14 Jun 2019
Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking Ziqi Yang Hung Dang E. Chang AAML 32 34 0 14 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 52 327 0 13 Jun 2019
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian Samet Oymak Zalan Fabian Mingchen Li Mahdi Soltanolkotabi MLT 28 89 0 12 Jun 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 66 487 0 12 Jun 2019
Parameterized Structured Pruning for Deep Neural Networks Günther Schindler Wolfgang Roth Franz Pernkopf Holger Froening 26 6 0 12 Jun 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 34 230 0 11 Jun 2019
Stable Rank Normalization for Improved Generalization in Neural Networks and GANs Amartya Sanyal Philip Torr P. Dokania 46 44 0 11 Jun 2019
Characterizing the implicit bias via a primal-dual analysis Ziwei Ji Matus Telgarsky 20 19 0 11 Jun 2019
Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization Navid Azizan Sahin Lale B. Hassibi 33 71 0 10 Jun 2019
Learning to Segment Skin Lesions from Noisy Annotations Z. Mirikharaji Yiqi Yan Ghassan Hamarneh 38 77 0 10 Jun 2019
The Generalization-Stability Tradeoff In Neural Network Pruning Brian Bartoldson Ari S. Morcos Adrian Barbu G. Erlebacher 38 73 0 09 Jun 2019
Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation P. Mitra 26 50 0 09 Jun 2019
The Implicit Bias of AdaGrad on Separable Data Qian Qian Xiaoyuan Qian 37 23 0 09 Jun 2019
Audio tagging with noisy labels and minimal supervision Eduardo Fonseca Manoj Plakal F. Font D. Ellis Xavier Serra 28 92 0 07 Jun 2019
Disentangling neural mechanisms for perceptual grouping Junkyung Kim Drew Linsley Kalpit C. Thakkar Thomas Serre OCL 43 54 0 04 Jun 2019
Dimensionality compression and expansion in Deep Neural Networks Stefano Recanatesi M. Farrell Madhu S. Advani Timothy Moore Guillaume Lajoie E. Shea-Brown 31 73 0 02 Jun 2019
The Principle of Unchanged Optimality in Reinforcement Learning Generalization A. Irpan Xingyou Song OffRL 41 7 0 02 Jun 2019
Are Anchor Points Really Indispensable in Label-Noise Learning? Xiaobo Xia Tongliang Liu N. Wang Bo Han Chen Gong Gang Niu Masashi Sugiyama NoLa 38 373 0 01 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 52 493 0 31 May 2019
On Network Design Spaces for Visual Recognition Ilija Radosavovic Justin Johnson Saining Xie Wan-Yen Lo Piotr Dollár 27 135 0 30 May 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 42 384 0 30 May 2019
Implicit Regularization of Accelerated Methods in Hilbert Spaces Nicolò Pagliana Lorenzo Rosasco 34 18 0 30 May 2019
Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models Amrita Roy Chowdhury Theodoros Rekatsinas S. Jha 28 10 0 30 May 2019
Educating Text Autoencoders: Latent Representation Guidance via Denoising T. Shen Jonas W. Mueller Regina Barzilay Tommi Jaakkola 19 4 0 29 May 2019
Generalization bounds for deep convolutional neural networks Philip M. Long Hanie Sedghi MLT 42 90 0 29 May 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 40 210 0 29 May 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 44 5 0 29 May 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 44 237 0 28 May 2019
Learning to Auto Weight: Entirely Data-driven and Highly Efficient Weighting Framework Zhenmao Li Yichao Wu Ken Chen Yudong Wu Shunfeng Zhou Jiaheng Liu Junjie Yan 21 5 0 27 May 2019
Combating Label Noise in Deep Learning Using Abstention S. Thulasidasan Tanmoy Bhattacharya J. Bilmes Gopinath Chennupati J. Mohd-Yusof NoLa 22 179 0 27 May 2019
Interpretable deep Gaussian processes with moments Chi-Ken Lu Scott Cheng-Hsin Yang Xiaoran Hao Patrick Shafto 36 19 0 27 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 53 124 0 27 May 2019
Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets Guy Hacohen Leshem Choshen D. Weinshall AI4TS OOD 22 56 0 26 May 2019
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks Yaoyu Zhang Zhi-Qin John Xu Yaoyu Zhang Zheng Ma MLT AI4CE 44 38 0 24 May 2019
Curriculum Loss: Robust Learning and Generalization against Label Corruption Yueming Lyu Ivor W. Tsang NoLa 71 172 0 24 May 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Simon Lacoste-Julien ODL 56 206 0 24 May 2019
Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network Oscar Chang Yuling Yao David Williams-King Hod Lipson BDL UQCV 39 8 0 23 May 2019