Visualizing the Loss Landscape of Neural Nets

28 December 2017

Hao Li

Papers citing "Visualizing the Loss Landscape of Neural Nets"

50 / 1,039 papers shown

Title
Luck Matters: Understanding Training Dynamics of Deep ReLU Networks Yuandong Tian Tina Jiang Qucheng Gong Ari S. Morcos 11 24 0 31 May 2019
Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence Aditya Golatkar Alessandro Achille Stefano Soatto 20 95 0 30 May 2019
Where is the Information in a Deep Neural Network? Alessandro Achille Giovanni Paolini Stefano Soatto 8 81 0 29 May 2019
SignalTrain: Profiling Audio Compressors with Deep Neural Networks Scott H. Hawley Benjamin Colburn S. I. Mimilakis 6 12 0 28 May 2019
Abstraction Mechanisms Predict Generalization in Deep Neural Networks Alex Gain H. Siegelmann AI4CE 11 6 0 27 May 2019
How degenerate is the parametrization of neural networks with the ReLU activation function? Julius Berner Dennis Elbrächter Philipp Grohs ODL 17 28 0 23 May 2019
LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning Huaiyu Li Weiming Dong Xing Mei Chongyang Ma Feiyue Huang Bao-Gang Hu OffRL 16 98 0 15 May 2019
Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment Chen Huang Shuangfei Zhai Walter A. Talbott Miguel Angel Bautista Shi Sun Carlos Guestrin J. Susskind 16 74 0 15 May 2019
Improving Model Training by Periodic Sampling over Weight Distributions S. Tripathi Jiayi Liu Unmesh Kurup Mohak Shah Sauptik Dhar 11 0 0 14 May 2019
The sharp, the flat and the shallow: Can weakly interacting agents learn to escape bad minima? N. Kantas P. Parpas G. Pavliotis ODL 11 8 0 10 May 2019
S4L: Self-Supervised Semi-Supervised Learning Xiaohua Zhai Avital Oliver Alexander Kolesnikov Lucas Beyer SSL VLM 21 787 0 09 May 2019
SinReQ: Generalized Sinusoidal Regularization for Low-Bitwidth Deep Quantized Training Ahmed T. Elthakeb Prannoy Pilligundla H. Esmaeilzadeh MQ 15 9 0 04 May 2019
Meta-learners' learning dynamics are unlike learners' Neil C. Rabinowitz OffRL 17 16 0 03 May 2019
On Expected Accuracy Ozan Irsoy UQCV 6 2 0 01 May 2019
Adversarial Training for Free! Ali Shafahi Mahyar Najibi Amin Ghiasi Zheng Xu John P. Dickerson Christoph Studer L. Davis Gavin Taylor Tom Goldstein AAML 13 1,224 0 29 Apr 2019
Forecasting in Big Data Environments: an Adaptable and Automated Shrinkage Estimation of Neural Networks (AAShNet) Ali Habibnia E. Maasoumi 6 5 0 25 Apr 2019
Knowledge Distillation via Route Constrained Optimization Xiao Jin Baoyun Peng Yichao Wu Yu Liu Jiaheng Liu Ding Liang Junjie Yan Xiaolin Hu 10 169 0 19 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 25 136 0 10 Apr 2019
Convergence rates for the stochastic gradient descent method for non-convex objective functions Benjamin J. Fehrman Benjamin Gess Arnulf Jentzen 17 101 0 02 Apr 2019
Why ResNet Works? Residuals Generalize Fengxiang He Tongliang Liu Dacheng Tao 6 243 0 02 Apr 2019
Parabolic Approximation Line Search for DNNs Max Mutschler A. Zell ODL 6 20 0 28 Mar 2019
TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications Frederik Heber Zofia Trstanova B. Leimkuhler 6 0 0 20 Mar 2019
Traversing the noise of dynamic mini-batch sub-sampled loss functions: A visual guide D. Kafka D. Wilke 13 0 0 20 Mar 2019
IMEXnet: A Forward Stable Deep Neural Network E. Haber Keegan Lensink Eran Treister Lars Ruthotto 30 40 0 06 Mar 2019
Positively Scale-Invariant Flatness of ReLU Neural Networks Mingyang Yi Qi Meng Wei-neng Chen Zhi-Ming Ma Tie-Yan Liu 13 17 0 06 Mar 2019
Generalisation in fully-connected neural networks for time series forecasting Anastasia Borovykh C. Oosterlee S. Bohté OOD AI4TS 14 3 0 14 Feb 2019
Task2Vec: Task Embedding for Meta-Learning Alessandro Achille Michael Lam Rahul Tewari Avinash Ravichandran Subhransu Maji Charless C. Fowlkes Stefano Soatto Pietro Perona SSL 12 309 0 10 Feb 2019
Improved Knowledge Distillation via Teacher Assistant Seyed Iman Mirzadeh Mehrdad Farajtabar Ang Li Nir Levine Akihiro Matsukawa H. Ghasemzadeh 22 1,065 0 09 Feb 2019
A Simple Baseline for Bayesian Uncertainty in Deep Learning Wesley J. Maddox T. Garipov Pavel Izmailov Dmitry Vetrov A. Wilson BDL UQCV 11 793 0 07 Feb 2019
A Scale Invariant Flatness Measure for Deep Network Minima Akshay Rangamani Nam H. Nguyen Abhishek Kumar Dzung Phan Sang H. Chin T. Tran ODL 20 31 0 06 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 12 147 0 02 Feb 2019
Compressing GANs using Knowledge Distillation Angeline Aguinaldo Ping Yeh-Chiang Alex Gain Ameya D. Patil Kolten Pearson S. Feizi GAN 11 83 0 01 Feb 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 14 314 0 29 Jan 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 11 490 0 28 Jan 2019
Visualized Insights into the Optimization Landscape of Fully Convolutional Networks Jianjie Lu K. Tong 17 12 0 20 Jan 2019
Ensemble Feature for Person Re-Identification Jiabao Wang Yang Li Zhuang Miao OOD 3DPC 23 1 0 17 Jan 2019
Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysis Yusuke Tsuzuku Issei Sato Masashi Sugiyama 14 74 0 15 Jan 2019
Neumann Networks for Inverse Problems in Imaging Davis Gilton Greg Ongie Rebecca Willett 6 24 0 13 Jan 2019
Multi-class Classification without Multi-class Labels Yen-Chang Hsu Zhaoyang Lv Joel Schlosser Phillip Odom Z. Kira 8 164 0 02 Jan 2019
LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks S. K. Roy Suvojit Manna S. Dubey B. B. Chaudhuri 11 49 0 01 Jan 2019
Precision Highway for Ultra Low-Precision Quantization Eunhyeok Park Dongyoung Kim S. Yoo Peter Vajda MQ AI4TS 8 12 0 24 Dec 2018
Kalman-based Spectro-Temporal ECG Analysis using Deep Convolutional Networks for Atrial Fibrillation Detection Zheng Zhao Simo Särkkä Ali Bahrami Rad 16 30 0 12 Dec 2018
Wireless Network Intelligence at the Edge Jihong Park S. Samarakoon M. Bennis Mérouane Debbah 16 518 0 07 Dec 2018
Deep learning for pedestrians: backpropagation in CNNs L. Boué 3DV PINN 11 4 0 29 Nov 2018
Understanding the impact of entropy on policy optimization Zafarali Ahmed Nicolas Le Roux Mohammad Norouzi Dale Schuurmans 6 225 0 27 Nov 2018
Sequentially Aggregated Convolutional Networks Yiwen Huang Rihui Wu Pinglai Ou Ziyong Feng 11 1 0 27 Nov 2018
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks Shuxuan Guo J. Álvarez Mathieu Salzmann 13 77 0 26 Nov 2018
Forward Stability of ResNet and Its Variants Linan Zhang Hayden Schaeffer 17 47 0 24 Nov 2018
Analytic Network Learning Kar-Ann Toh 6 9 0 20 Nov 2018
Characterizing Well-Behaved vs. Pathological Deep Neural Networks Mitchell Stern 8 0 0 07 Nov 2018