How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 167 papers shown

Title
Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization G. Zhang S. Fattahi Richard Y. Zhang 45 36 0 13 Apr 2025
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management Yi-Hu Feng Tianlin Li Tian Xie 62 0 0 26 Feb 2025
Attribute Inference Attacks for Federated Regression Tasks Francesco Diana Othmane Marfoq Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML 204 1 0 19 Nov 2024
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 150 0 0 10 Oct 2024
Automatically Adaptive Conformal Risk Control Vincent Blot Anastasios Nikolas Angelopoulos Michael I Jordan Nicolas Brunel AI4CE 44 2 0 25 Jun 2024
3D Gaussian Splatting as Markov Chain Monte Carlo Shakiba Kheradmand Daniel Rebain Gopal Sharma Weiwei Sun Jeff Tseng Hossam N. Isack Abhishek Kar Andrea Tagliasacchi Kwang Moo Yi 3DGS 50 49 0 15 Apr 2024
Training-set-free two-stage deep learning for spectroscopic data de-noising Dongchen Huang Junde Liu Tian Qian Hongming Weng 36 0 0 29 Feb 2024
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 45 0 0 08 Feb 2024
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 42 2 0 20 Nov 2023
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 30 8 0 24 Oct 2023
Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled Gradient Descent, Even with Overparameterization Cong Ma Xingyu Xu Tian Tong Yuejie Chi 18 9 0 09 Oct 2023
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization Nuoya Xiong Lijun Ding Simon S. Du 35 11 0 03 Oct 2023
Symmetry & Critical Points for Symmetric Tensor Decomposition Problems Yossi Arjevani Gal Vinograd 31 5 0 13 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 32 6 0 25 May 2023
Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays Adrian Redder Arunselvan Ramaswamy Holger Karl 20 1 0 11 May 2023
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning Yulai Zhao Zhuoran Yang Zhaoran Wang Jason D. Lee 43 3 0 08 May 2023
Learning Rate Schedules in the Presence of Distribution Shift Matthew Fahrbach Adel Javanmard Vahab Mirrokni Pratik Worah 24 6 0 27 Mar 2023
Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks Arun Ganesh Daogao Liu Sewoong Oh Abhradeep Thakurta ODL 27 12 0 20 Feb 2023
Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption Jun Liu Ye Yuan ODL 16 1 0 15 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 29 1 0 28 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 23 13 0 19 Jan 2023
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data Harsh Rangwani Sumukh K Aithal Mayank Mishra R. Venkatesh Babu 31 28 0 28 Dec 2022
Escaping From Saddle Points Using Asynchronous Coordinate Gradient Descent Marco Bornstein Jin-Peng Liu Jingling Li Furong Huang 21 0 0 17 Nov 2022
Neural Langevin Dynamics: towards interpretable Neural Stochastic Differential Equations Simon Koop M. Peletier J. Portegies Vlado Menkovski DiffM 35 1 0 17 Nov 2022
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models Shujian Zhang Chengyue Gong Xingchao Liu RALM 49 6 0 02 Nov 2022
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points R. Tappenden Martin Takáč 15 0 0 02 Nov 2022
Local Model Reconstruction Attacks in Federated Learning and their Uses Ilias Driouich Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML FedML 32 2 0 28 Oct 2022
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 167 68 0 27 Oct 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 35 7 0 04 Oct 2022
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition Jianhao Ma Li-Zhen Guo S. Fattahi 38 4 0 01 Oct 2022
Nonconvex Matrix Factorization is Geodesically Convex: Global Landscape Analysis for Fixed-rank Matrix Optimization From a Riemannian Perspective Yuetian Luo Nicolas García Trillos 24 6 0 29 Sep 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 24 17 0 29 Sep 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 25 41 0 19 Sep 2022
Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion G. Zhang Hong-Ming Chiu Richard Y. Zhang 24 10 0 24 Aug 2022
CoShNet: A Hybrid Complex Valued Neural Network using Shearlets Manny Ko Ujjawal K. Panchal Héctor Andrade-Loarca Andres Mendez-Vazquez 27 1 0 14 Aug 2022
A Gradient Smoothed Functional Algorithm with Truncated Cauchy Random Perturbations for Stochastic Optimization Akash Mondal A. PrashanthL. S. Bhatnagar 17 2 0 30 Jul 2022
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time Jun-Kun Wang Andre Wibisono 30 9 0 05 Jul 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 29 10 0 08 Jun 2022
Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification G. Zhang S. Fattahi Richard Y. Zhang 50 23 0 07 Jun 2022
Subspace Phase Retrieval Meng Xu Dekuan Dong J. Wang 19 2 0 06 Jun 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 14 8 0 30 May 2022
Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing Keisuke Suzuki AI4CE 30 0 0 25 May 2022
Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimization Keisuke Suzuki 25 5 0 24 May 2022
Estimation and Inference by Stochastic Optimization Jean-Jacques Forneron 22 5 0 06 May 2022
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 27 7 0 05 May 2022
Gradient Descent, Stochastic Optimization, and Other Tales Jun Lu 14 8 0 02 May 2022
Accelerating nuclear-norm regularized low-rank matrix optimization through Burer-Monteiro decomposition Ching-pei Lee Ling Liang Tianyun Tang Kim-Chuan Toh 27 11 0 29 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 24 109 0 28 Feb 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 28 8 0 18 Feb 2022