Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Neural Information Processing Systems (NeurIPS), 2014

10 June 2014

Papers citing "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization"

50 / 631 papers shown

Title
Myriad: a real-world testbed to bridge trajectory optimization and deep learningNeural Information Processing Systems (NeurIPS), 2022 Nikolaus H. R. Howe Simon Dufort-Labbé Nitarshan Rajkumar Pierre-Luc Bacon 140 5 0 22 Feb 2022
How Do Vision Transformers Work?International Conference on Learning Representations (ICLR), 2022 Namuk Park Songkuk Kim ViT 417 594 0 14 Feb 2022
Efficiently Escaping Saddle Points in Bilevel Optimization Minhui Huang Xuxing Chen Kaiyi Ji Shiqian Ma Lifeng Lai 221 28 0 08 Feb 2022
When Do Flat Minima Optimizers Work?Neural Information Processing Systems (NeurIPS), 2022 Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 490 85 0 01 Feb 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 163 11 0 31 Jan 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order OptimizationInternational Conference on Machine Learning (ICML), 2022 Frederik Benzing ODL 268 29 0 28 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization LandscapeInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2022 Devansh Bisla Jing Wang A. Choromańska 279 45 0 20 Jan 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 214 1 0 03 Jan 2022
Forecasting Brain Activity Based on Models of Spatio-Temporal Brain Dynamics: A Comparison of Graph Neural Network Architectures S. Wein Alina Schüller A. Tomé W. Malloni M. Greenlee E. Lang AI4CE 189 16 0 08 Dec 2021
Saliency Diversified Deep Ensemble for Robustness to Adversaries Alexander A. Bogun Dimche Kostadinov Damian Borth AAML FedML 105 5 0 07 Dec 2021
Boosting Unsupervised Domain Adaptation with Soft Pseudo-label and Curriculum Learning Shengjia Zhang Tiancheng Lin Yi Tian Xu 186 6 0 03 Dec 2021
The (1+1)-ES Reliably Overcomes Saddle Points Tobias Glasmachers 90 0 0 01 Dec 2021
Escape saddle points by a simple gradient-descent based algorithmNeural Information Processing Systems (NeurIPS), 2021 Chenyi Zhang Tongyang Li ODL 140 15 0 28 Nov 2021
NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning Buyun Liang Tim Mitchell Ju Sun 207 5 0 27 Nov 2021
Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stoppingJournal of machine learning research (JMLR), 2021 Xuran Meng Jianfeng Yao 267 10 0 26 Nov 2021
Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion Nobuhiko Wakai Satoshi Sato Yasunori Ishii Takayoshi Yamashita 137 12 0 25 Nov 2021
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor RectificationInternational Conference on Machine Learning (ICML), 2021 L. Pan Longbo Huang Tengyu Ma Huazhe Xu OffRL OnRL 323 69 0 22 Nov 2021
ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation GuaranteesNeural Information Processing Systems (NeurIPS), 2021 Kuan-Lin Chen Ching-Hua Lee H. Garudadri Bhaskar D. Rao AI4TS 258 7 0 10 Nov 2021
Inertial Newton Algorithms Avoiding Strict Saddle Points Camille Castera ODL 149 4 0 08 Nov 2021
PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations Louis Fortier-Dubois Gaël Letarte Benjamin Leblanc Franccois Laviolette Pascal Germain UQCV 236 1 0 28 Oct 2021
Optimal Auction Design for the Gradual Procurement of Strategic Service Provider Agents F. Farhadi Maria Chli N. Jennings 48 0 0 25 Oct 2021
Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning Uche M. Osahor Nasser M. Nasrabadi 128 14 0 18 Oct 2021
A Cubic Regularization Approach for Finding Local Minimax Points in Nonconvex Minimax Optimization Ziyi Chen Zhengyang Hu Qunwei Li Zhe Wang Yi Zhou 282 8 0 14 Oct 2021
Improving Adversarial Robustness for Free with Snapshot Ensemble Yihao Wang AAML UQCV 128 1 0 07 Oct 2021
Boost Neural Networks by Checkpoints Feng Wang Gu-Yeon Wei Qiao Liu Jinxiang Ou Xian Wei Hairong Lv FedML UQCV 147 12 0 03 Oct 2021
Variational learning of quantum ground states on spiking neuromorphic hardware Robert Klassert A. Baumbach Mihai A. Petrovici M. Gärttner 161 9 0 30 Sep 2021
Scale-invariant Learning by Physics Inversion Philipp Holl V. Koltun Nils Thuerey PINN AI4CE 221 9 0 30 Sep 2021
Generalisations and improvements of New Q-Newton's method Backtracking T. Truong 59 0 0 23 Sep 2021
Neural forecasting at scale Philippe Chatigny Shengrui Wang Jean-Marc Patenaude and Boris N. Oreshkin AI4TS 195 1 0 20 Sep 2021
A Continuous Optimisation Benchmark Suite from Neural Network Regression K. Malan C. Cleghorn ODL 80 2 0 12 Sep 2021
A Neural Tangent Kernel Perspective of Infinite Tree EnsemblesInternational Conference on Learning Representations (ICLR), 2021 Ryuichi Kanoh M. Sugiyama 79 7 0 10 Sep 2021
An Introduction to Hamiltonian Monte Carlo Method for Sampling Nisheeth K. Vishnoi 105 14 0 27 Aug 2021
New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation T. Truong 74 5 0 23 Aug 2021
Towards Understanding Theoretical Advantages of Complex-Reaction Networks Shao-Qun Zhang Gaoxin Wei Zhi Zhou 202 20 0 15 Aug 2021
Expressive Power and Loss Surfaces of Deep Learning Models S. Dube 128 0 0 08 Aug 2021
Sparse Bayesian Deep Learning for Dynamic System Identification Hongpeng Zhou Chahine Ibrahim W. Zheng Wei Pan BDL 138 33 0 27 Jul 2021
Taxonomizing local versus global structure in neural network loss landscapesNeural Information Processing Systems (NeurIPS), 2021 Yaoqing Yang Liam Hodgkinson Ryan Theisen Joe Zou Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney 355 43 0 23 Jul 2021
Estimation of a regression function on a manifold by fully connected deep neural networksJournal of Statistical Planning and Inference (JSPI), 2021 Michael Kohler S. Langer U. Reif 169 7 0 20 Jul 2021
How many degrees of freedom do we need to train deep networks: a loss landscape perspective Brett W. Larsen Stanislav Fort Nico Becker Surya Ganguli UQCV 204 29 0 13 Jul 2021
Activated Gradients for Deep Neural NetworksIEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2021 Mei Liu Liangming Chen Xiaohao Du Long Jin Mingsheng Shang ODL AI4CE 157 182 0 09 Jul 2021
Immunization of Pruning Attack in DNN Watermarking Using Constant Weight Code Minoru Kuribayashi Tatsuya Yasui Asad U. Malik N. Funabiki AAML 74 2 0 07 Jul 2021
Post-Selections in AI and How to Avoid Them J. Weng 130 1 0 19 Jun 2021
Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic ModelsNeural Information Processing Systems (NeurIPS), 2021 Courtney Paquette Elliot Paquette ODL 157 17 0 07 Jun 2021
Control-Oriented Model-Based Reinforcement Learning with Implicit DifferentiationAAAI Conference on Artificial Intelligence (AAAI), 2021 Evgenii Nikishin Romina Abachi Rishabh Agarwal Pierre-Luc Bacon OffRL 186 42 0 06 Jun 2021
Escaping Saddle Points Faster with Stochastic MomentumInternational Conference on Learning Representations (ICLR), 2020 Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy ODL 171 24 0 05 Jun 2021
Solving hybrid machine learning tasks by traversing weight space geodesics G. Raghavan Matt Thomson 86 0 0 05 Jun 2021
A Scalable Second Order Method for Ill-Conditioned Matrix Completion from Few SamplesInternational Conference on Machine Learning (ICML), 2021 C. Kümmerle C. M. Verdun 162 24 0 03 Jun 2021
Discovering Diverse Nearly Optimal Policies with Successor Features Tom Zahavy Brendan O'Donoghue André Barreto Volodymyr Mnih Sebastian Flennerhag Satinder Singh 150 24 0 01 Jun 2021
Search Spaces for Neural Model Training Darko Stosic Dusan Stosic 139 4 0 27 May 2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and InvariancesInternational Conference on Machine Learning (ICML), 2021 Berfin cSimcsek François Ged Arthur Jacot Francesco Spadaro Clément Hongler W. Gerstner Johanni Brea AI4CE 270 118 0 25 May 2021