Newton Sketch: A Linear-time Optimization Algorithm with Linear-Quadratic Convergence

9 May 2015

Papers citing "Newton Sketch: A Linear-time Optimization Algorithm with Linear-Quadratic Convergence"

50 / 131 papers shown

Title
Non-Asymptotic Optimization and Generalization Bounds for Stochastic Gauss-Newton in Overparameterized Models Semih Cayci 88 0 0 06 Nov 2025
Sequential Least-Squares Estimators with Fast Randomized Sketching for Linear Statistical Models Guan-Yu Chen Xi Yang 40 0 0 08 Sep 2025
Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees A. Agafonov Vladislav Ryspayev Samuel Horváth Alexander V. Gasnikov Martin Takáč Slavomír Hanzely 60 1 0 27 Aug 2025
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning Jingruo Sun Zachary Frangella Madeleine Udell 178 2 0 28 Jan 2025
CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural NetworksNeural Information Processing Systems (NeurIPS), 2024 Miria Feng Zachary Frangella Mert Pilanci BDL 364 2 0 02 Nov 2024
Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and DebiasingInternational Conference on Learning Representations (ICLR), 2024 Elad Romanov Fangzhao Zhang Mert Pilanci 134 1 0 02 Oct 2024
FLeNS: Federated Learning with Enhanced Nesterov-Newton SketchBigData Congress [Services Society] (BSS), 2024 Sunny Gupta Mohit Jindal Pankhi Kashyap Pranav Jeevan Amit Sethi FedML 202 0 0 23 Sep 2024
Incremental Gauss--Newton Methods with Superlinear Convergence Rates Zhiling Zhou Zhuanghua Liu Chengchang Liu Luo Luo 151 1 0 03 Jul 2024
Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning Michał Dereziński Michael W. Mahoney 232 18 0 17 Jun 2024
Stochastic Newton Proximal Extragradient Method Ruichen Jiang Michal Dereziñski Aryan Mokhtari 92 0 0 03 Jun 2024
Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients Sachin Garg A. Berahas Michal Dereziñski 181 2 0 23 Apr 2024
Gradient Descent is Pareto-Optimal in the Oracle Complexity and Memory Tradeoff for Feasibility ProblemsIEEE Annual Symposium on Foundations of Computer Science (FOCS), 2024 Moise Blanchard 212 1 0 10 Apr 2024
Stochastic Online Optimization for Cyber-Physical and Robotic Systems Hao Ma Melanie Zeilinger Michael Muehlebach 182 3 0 08 Apr 2024
Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms Antoine Godichon-Baggioni Wei Lu Bruno Portier 476 3 0 15 Jan 2024
FedNS: A Fast Sketching Newton-Type Algorithm for Federated Learning Jian Li Yong Liu Wei Wang Haoran Wu Weiping Wang FedML 229 5 0 05 Jan 2024
Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applications Yifan Zhang Joe Kileel 351 6 0 09 Nov 2023
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity Pengyun Yue Hanzheng Zhao Cong Fang Di He Liwei Wang Zhouchen Lin Song-Chun Zhu 176 1 0 23 Sep 2023
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale Hao-Jun Michael Shi Tsung-Hsien Lee Shintaro Iwasaki Jose Gallego-Posada Zhijing Li Kaushik Rangadurai Dheevatsa Mudigere Michael Rabbat ODL 200 43 0 12 Sep 2023
Surrogate-based Autotuning for Randomized Sketching Algorithms in Regression Problems Younghyun Cho James Demmel Michal Derezinski Haoyun Li Hengrui Luo Michael W. Mahoney Riley Murray 195 7 0 30 Aug 2023
Gradient Coding with Iterative Block Leverage Score SamplingIEEE Transactions on Information Theory (IEEE Trans. Inf. Theory), 2023 Neophytos Charalambides Mert Pilanci Alfred Hero 257 4 0 06 Aug 2023
Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Rate Zhan Gao Aryan Mokhtari Alec Koppel 114 2 0 27 Jun 2023
Memory-Query Tradeoffs for Randomized Convex OptimizationIEEE Annual Symposium on Foundations of Computer Science (FOCS), 2023 Xinyu Chen Binghui Peng 180 7 0 21 Jun 2023
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative SketchingInternational Conference on Machine Learning (ICML), 2023 Ilgee Hong Sen Na Michael W. Mahoney Mladen Kolar 148 6 0 28 May 2023
Sharpened Lazy Incremental Quasi-Newton MethodInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Aakash Lahoti Spandan Senapati K. Rajawat Alec Koppel 311 2 0 26 May 2023
ISAAC Newton: Input-based Approximate Curvature for Newton's MethodInternational Conference on Learning Representations (ICLR), 2023 Felix Petersen Tobias Sutter Christian Borgelt Dongsung Huh Hilde Kuehne Yuekai Sun Oliver Deussen ODL 187 5 0 01 May 2023
Robust, randomized preconditioning for kernel ridge regression Mateo Díaz Ethan N. Epperly Zachary Frangella J. Tropp R. Webber 463 16 0 24 Apr 2023
SketchySGD: Reliable Stochastic Optimization via Randomized Curvature EstimatesSIAM Journal on Mathematics of Data Science (SIMODS), 2022 Zachary Frangella Pratik Rathore Shipu Zhao Madeleine Udell 310 8 0 16 Nov 2022
A Randomised Subspace Gauss-Newton Method for Nonlinear Least-Squares C. Cartis J. Fowkes Zhen Shao 60 17 0 10 Nov 2022
Explicit Second-Order Min-Max Optimization: Practical Algorithms and Complexity Analysis Tianyi Lin P. Mertikopoulos Michael I. Jordan 424 14 0 23 Oct 2022
DRSOM: A Dimension Reduced Second-Order Method Chuwen Zhang Dongdong Ge Chang He Bo Jiang Yuntian Jiang Yi-Li Ye 126 7 0 30 Jul 2022
SP2: A Second Order Stochastic Polyak MethodInternational Conference on Learning Representations (ICLR), 2022 Shuang Li W. Swartworth Martin Takávc Deanna Needell Robert Mansel Gower 149 14 0 17 Jul 2022
Riemannian Natural Gradient Methods Jiang Hu Ruicheng Ao Anthony Man-Cho So Minghan Yang Zaiwen Wen 133 13 0 15 Jul 2022
Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches Michal Derezinski 295 11 0 06 Jun 2022
Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programming Sen Na Michael W. Mahoney 329 10 0 27 May 2022
Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation M. Morshed ODL 139 1 0 23 May 2022
Hessian Averaging in Stochastic Newton Methods Achieves Superlinear ConvergenceMathematical programming (Math. Program.), 2022 Sen Na Michal Derezinski Michael W. Mahoney 268 21 0 20 Apr 2022
Optimal Subsampling for High-dimensional Ridge RegressionKnowledge-Based Systems (KBS), 2022 Hanyu Li Cheng Niu 139 7 0 18 Apr 2022
Efficient Convex Optimization Requires Superlinear MemoryAnnual Conference Computational Learning Theory (COLT), 2022 A. Marsden Willie Neiswanger Aaron Sidford Gregory Valiant 172 16 0 29 Mar 2022
Operator Sketching for Deep Unrolling Networks Junqi Tang Subhadip Mukherjee Carola-Bibiane Schönlieb 207 0 0 21 Mar 2022
Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower BoundsIEEE Transactions on Information Theory (IEEE Trans. Inf. Theory), 2022 Burak Bartan Mert Pilanci 115 6 0 18 Mar 2022
Accelerating Plug-and-Play Image Reconstruction via Multi-Stage Sketched Gradients Junqi Tang 163 2 0 14 Mar 2022
pylspack: Parallel algorithms and data structures for sketching, column subset selection, regression and leverage scoresACM Transactions on Mathematical Software (TOMS), 2022 Aleksandros Sobczyk Efstratios Gallopoulos 169 9 0 05 Mar 2022
Efficient Privacy Preserving Logistic Regression for Horizontally Distributed Data G. Miao 100 0 0 05 Feb 2022
The Complexity of Dynamic Least-Squares RegressionIEEE Annual Symposium on Foundations of Computer Science (FOCS), 2022 Shunhua Jiang Binghui Peng Omri Weinstein 169 6 0 01 Jan 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 308 67 0 14 Dec 2021
Learning Linear Models Using Distributed Iterative Hessian Sketching Han Wang James Anderson 195 2 0 08 Dec 2021
Learning Augmentation Distributions using Transformed Risk Minimization Evangelos Chatzipantazis Stefanos Pertigkiozoglou Kostas Daniilidis Guang Cheng 212 15 0 16 Nov 2021
Nys-Newton: Nyström-Approximated Curvature for Stochastic Optimization Dinesh Singh Hardik Tankaria M. Yamada ODL 215 3 0 16 Oct 2021
Computational Polarization: An Information-theoretic Method for Resilient ComputingIEEE Transactions on Information Theory (IEEE Trans. Inf. Theory), 2021 Mert Pilanci 149 3 0 08 Sep 2021
Functional Principal Subspace Sampling for Large Scale Functional Data AnalysisElectronic Journal of Statistics (EJS), 2021 Shiyuan He Xiaomeng Yan 196 5 0 08 Sep 2021