Debiasing Distributed Second Order Optimization with Surrogate Sketching
and Scaled Regularization

Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization

2 July 2020

Michal Derezinski

Michael W. Mahoney

ArXiv (abs)PDF HTML

Papers citing "Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization"

14 / 14 papers shown

Title
Surrogate-based Autotuning for Randomized Sketching Algorithms in Regression Problems Younghyun Cho James Demmel Michal Derezinski Haoyun Li Hengrui Luo Michael W. Mahoney Riley Murray 70 7 0 30 Aug 2023
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching Ilgee Hong Sen Na Michael W. Mahoney Mladen Kolar 58 5 0 28 May 2023
Asymptotics of the Sketched Pseudoinverse Daniel LeJeune Pratik V. Patil Hamid Javadi Richard G. Baraniuk Robert Tibshirani 54 10 0 07 Nov 2022
Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches Michal Derezinski 130 6 0 06 Jun 2022
Hessian Averaging in Stochastic Newton Methods Achieves Superlinear Convergence Sen Na Michal Derezinski Michael W. Mahoney 118 17 0 20 Apr 2022
Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Bounds Burak Bartan Mert Pilanci 26 5 0 18 Mar 2022
Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update Michal Derezinski Jonathan Lacotte Mert Pilanci Michael W. Mahoney 119 28 0 15 Jul 2021
Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionality Jonathan Lacotte Yifei Wang Mert Pilanci 67 17 0 15 May 2021
Adaptive and Oblivious Randomized Subspace Methods for High-Dimensional Optimization: Sharp Analysis and Lower Bounds Jonathan Lacotte Mert Pilanci 107 12 0 13 Dec 2020
Sparse sketches with small inversion bias Michal Derezinski Zhenyu Liao Yan Sun Michael W. Mahoney 107 22 0 21 Nov 2020
Sampling from a $k$ -DPP without looking at all items Daniele Calandriello Michal Derezinski Michal Valko 61 24 0 30 Jun 2020
Precise expressions for random projections: Low-rank approximation and randomized Newton Michal Derezinski Feynman T. Liang Zhenyu A. Liao Michael W. Mahoney 84 24 0 18 Jun 2020
Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimization Jonathan Lacotte Mert Pilanci 55 24 0 10 Jun 2020
Exact expressions for double descent and implicit regularization via surrogate random design Michal Derezinski Feynman T. Liang Michael W. Mahoney 82 78 0 10 Dec 2019