Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

10 June 2014

Papers citing "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization"

50 / 251 papers shown

Title
Boosting Unsupervised Domain Adaptation with Soft Pseudo-label and Curriculum Learning Shengjia Zhang Tiancheng Lin Yi Tian Xu 30 5 0 03 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 31 15 0 28 Nov 2021
NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning Buyun Liang Tim Mitchell Ju Sun 17 3 0 27 Nov 2021
Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping Xuran Meng Jianfeng Yao 29 7 0 26 Nov 2021
Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion Nobuhiko Wakai Satoshi Sato Yasunori Ishii Takayoshi Yamashita 19 8 0 25 Nov 2021
Inertial Newton Algorithms Avoiding Strict Saddle Points Camille Castera ODL 15 2 0 08 Nov 2021
PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations Louis Fortier-Dubois Gaël Letarte Benjamin Leblanc Franccois Laviolette Pascal Germain UQCV 19 0 0 28 Oct 2021
Improving Adversarial Robustness for Free with Snapshot Ensemble Yihao Wang AAML UQCV 17 1 0 07 Oct 2021
Boost Neural Networks by Checkpoints Feng Wang Gu-Yeon Wei Qiao Liu Jinxiang Ou Xian Wei Hairong Lv FedML UQCV 29 10 0 03 Oct 2021
Variational learning of quantum ground states on spiking neuromorphic hardware Robert Klassert A. Baumbach Mihai A. Petrovici M. Gärttner 28 7 0 30 Sep 2021
Generalisations and improvements of New Q-Newton's method Backtracking T. Truong 11 0 0 23 Sep 2021
Neural forecasting at scale Philippe Chatigny Shengrui Wang Jean-Marc Patenaude and Boris N. Oreshkin AI4TS 35 1 0 20 Sep 2021
New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation T. Truong 27 5 0 23 Aug 2021
Sparse Bayesian Deep Learning for Dynamic System Identification Hongpeng Zhou Chahine Ibrahim W. Zheng Wei Pan BDL 23 25 0 27 Jul 2021
Estimation of a regression function on a manifold by fully connected deep neural networks Michael Kohler S. Langer U. Reif 22 4 0 20 Jul 2021
Activated Gradients for Deep Neural Networks Mei Liu Liangming Chen Xiaohao Du Long Jin Mingsheng Shang ODL AI4CE 35 135 0 09 Jul 2021
Immunization of Pruning Attack in DNN Watermarking Using Constant Weight Code Minoru Kuribayashi Tatsuya Yasui Asad U. Malik N. Funabiki AAML 23 1 0 07 Jul 2021
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation Evgenii Nikishin Romina Abachi Rishabh Agarwal Pierre-Luc Bacon OffRL 52 35 0 06 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
Apply Artificial Neural Network to Solving Manpower Scheduling Problem Tianyu Liu Lingyu Zhang 17 1 0 07 May 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 29 10 0 19 Mar 2021
Panel semiparametric quantile regression neural network for electricity consumption forecasting Xingcai Zhou Jiangyan Wang AI4TS 19 16 0 01 Mar 2021
Learning Neural Network Subspaces Mitchell Wortsman Maxwell Horton Carlos Guestrin Ali Farhadi Mohammad Rastegari UQCV 27 85 0 20 Feb 2021
Appearance of Random Matrix Theory in Deep Learning Nicholas P. Baskerville Diego Granziol J. Keating 18 11 0 12 Feb 2021
Recent Advances in Adversarial Training for Adversarial Robustness Tao Bai Jinqi Luo Jun Zhao Bihan Wen Qian Wang AAML 86 476 0 02 Feb 2021
Physical deep learning based on optimal control of dynamical systems Genki Furuhata T. Niiyama S. Sunada PINN AI4CE 34 14 0 16 Dec 2020
A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization Adepu Ravi Sankar Yash Khasbage Rahul Vigneswaran V. Balasubramanian 25 42 0 07 Dec 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 29 2 0 15 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 23 41 0 26 Oct 2020
Softmax Deep Double Deterministic Policy Gradients Ling Pan Qingpeng Cai Longbo Huang 72 86 0 19 Oct 2020
Escaping Saddle Points in Ill-Conditioned Matrix Completion with a Scalable Second Order Method C. Kümmerle C. M. Verdun 19 6 0 07 Sep 2020
A community-powered search of machine learning strategy space to find NMR property prediction models Lars A. Bratholm W. Gerrard Brandon M. Anderson Shaojie Bai Sunghwan Choi ... A. Torrubia Devin Willmott C. Butts David R. Glowacki Kaggle participants 24 16 0 13 Aug 2020
Meta Continual Learning via Dynamic Programming R. Krishnan Prasanna Balaprakash CLL 22 10 0 05 Aug 2020
Reformulation of the No-Free-Lunch Theorem for Entangled Data Sets Kunal Sharma M. Cerezo Zoë Holmes L. Cincio A. Sornborger Patrick J. Coles 24 48 0 09 Jul 2020
Weak error analysis for stochastic gradient descent optimization algorithms A. Bercher Lukas Gonon Arnulf Jentzen Diyora Salimova 36 4 0 03 Jul 2020
Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications Yating Wang Wei Deng Guang Lin 26 13 0 29 Jun 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 35 19 0 19 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 37 49 0 16 Jun 2020
SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness Mohammadamin Tavakoli Forest Agostinelli Pierre Baldi AAML FAtt 36 39 0 16 Jun 2020
The Limit of the Batch Size Yang You Yuhui Wang Huan Zhang Zhao-jie Zhang J. Demmel Cho-Jui Hsieh 16 15 0 15 Jun 2020
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis Courtney Paquette B. V. Merrienboer Elliot Paquette Fabian Pedregosa 31 25 0 08 Jun 2020
On the Promise of the Stochastic Generalized Gauss-Newton Method for Training DNNs Matilde Gargiani Andrea Zanelli Moritz Diehl Frank Hutter ODL 14 18 0 03 Jun 2020
Review of Mathematical frameworks for Fairness in Machine Learning E. del Barrio Paula Gordaliza Jean-Michel Loubes FaML FedML 15 39 0 26 May 2020
DENS-ECG: A Deep Learning Approach for ECG Signal Delineation A. Peimankar S. Puthusserypady 38 119 0 18 May 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 36 13 0 23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Block Layer Decomposition schemes for training Deep Neural Networks L. Palagi R. Seccia 25 5 0 18 Mar 2020
On Isometry Robustness of Deep 3D Point Cloud Models under Adversarial Attacks Yue Zhao Yuwei Wu Caihua Chen A. Lim 3DPC 16 70 0 27 Feb 2020
On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping Sanghyun Hong Varun Chandrasekaran Yigitcan Kaya Tudor Dumitras Nicolas Papernot AAML 28 136 0 26 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 55 155 0 21 Feb 2020