v1v2 (latest)

A mathematical model for automatic differentiation in machine learning

3 June 2020

Papers citing "A mathematical model for automatic differentiation in machine learning"

37 / 37 papers shown

Title
Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks Thang Do Arnulf Jentzen Adrian Riekert 103 1 0 03 Mar 2025
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach Yequan Zhao Hai Li Ian Young Zheng Zhang MQ 97 3 0 07 Nov 2024
Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient Vu C. Dinh L. Ho Cuong V Nguyen 38 1 0 29 Oct 2024
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates Sheheryar Mehmood Peter Ochs 77 1 0 21 Oct 2024
Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization Nachuan Xiao Kuang-Yu Ding Xiaoyin Hu Kim-Chuan Toh 91 4 0 15 Apr 2024
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates Riccardo Grazzi Massimiliano Pontil Saverio Salzo 93 1 0 18 Mar 2024
On the numerical reliability of nonsmooth autodiff: a MaxPool case study Ryan Boustany 88 1 0 05 Jan 2024
The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks Lénaic Chizat Praneeth Netrapalli 148 4 0 30 Nov 2023
Piecewise Polynomial Regression of Tame Functions via Integer Programming Gilles Bareilles Johannes Aspman Jiri Nemecek Georgios Korpas 52 1 0 22 Nov 2023
Adam-family Methods with Decoupled Weight Decay in Deep Learning Kuang-Yu Ding Nachuan Xiao Kim-Chuan Toh 56 3 0 13 Oct 2023
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks Yequan Zhao Xinling Yu Zhixiong Chen Ziyue Liu Sijia Liu Zheng Zhang PINN 71 11 0 18 Aug 2023
Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning Yu-Shuen Tang Zhimin Ding Dimitrije Jankov Binhang Yuan Daniel Bourgeois C. Jermaine BDL 95 6 0 31 May 2023
Generalizing Adam to Manifolds for Efficiently Training Transformers B. Brantner MedIm 49 3 0 26 May 2023
Understanding Automatic Differentiation Pitfalls Jan Huckelheim Harshitha Menon William S. Moses Bruce Christianson P. Hovland Laurent Hascoet PINN 58 4 0 12 May 2023
$ω$ PAP Spaces: Reasoning Denotationally About Higher-Order, Recursive Probabilistic and Differentiable Programs Mathieu Huot Alexander K. Lew Vikash K. Mansinghka S. Staton 38 7 0 21 Feb 2023
On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters Wonyeol Lee Sejun Park A. Aiken PINN 44 6 0 31 Jan 2023
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems Jérôme Bolte Edouard Pauwels Antonio Silveti-Falls 63 13 0 15 Dec 2022
Variants of SGD for Lipschitz Continuous Loss Functions in Low-Precision Environments Michael R. Metel 58 1 0 09 Nov 2022
Lifted Bregman Training of Neural Networks Xiaoyu Wang Martin Benning 34 6 0 18 Aug 2022
Fixed-Point Automatic Differentiation of Forward--Backward Splitting Algorithms for Partly Smooth Functions Sheheryar Mehmood Peter Ochs 89 3 0 05 Aug 2022
Flexible Differentiable Optimization via Model Transformations Mathieu Besançon J. Garcia B. Legat Akshay Sharma 74 10 0 10 Jun 2022
On the complexity of nonsmooth automatic differentiation Jérôme Bolte Ryan Boustany Edouard Pauwels B. Pesquet-Popescu 52 2 0 01 Jun 2022
Automatic differentiation of nonsmooth iterative algorithms Jérôme Bolte Edouard Pauwels Samuel Vaiter 104 23 0 31 May 2022
Training invariances and the low-rank phenomenon: beyond linear networks Thien Le Stefanie Jegelka 82 33 0 28 Jan 2022
Path differentiability of ODE flows S. Marx Edouard Pauwels 60 2 0 11 Jan 2022
A Gradient Sampling Algorithm for Stratified Maps with Applications to Topological Data Analysis Jacob Leygonie Mathieu Carrière Théo Lacombe S. Oudot 41 9 0 01 Sep 2021
Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions Pascal Bianchi W. Hachem S. Schechtman 13 11 0 04 Aug 2021
Numerical influence of ReLU'(0) on backpropagation David Bertoin Jérôme Bolte Sébastien Gerchinovitz Edouard Pauwels 80 0 0 23 Jun 2021
Nonsmooth Implicit Differentiation for Machine Learning and Optimization Jérôme Bolte Tam Le Edouard Pauwels Antonio Silveti-Falls 88 57 0 08 Jun 2021
Implicit differentiation for fast hyperparameter selection in non-smooth convex learning Quentin Bertrand Quentin Klopfenstein Mathurin Massias Mathieu Blondel Samuel Vaiter Alexandre Gramfort Joseph Salmon 99 28 0 04 May 2021
Second-order step-size tuning of SGD for non-convex optimization Camille Castera Jérôme Bolte Cédric Févotte Edouard Pauwels ODL 41 10 0 05 Mar 2021
A General Descent Aggregation Framework for Gradient-based Bi-level Optimization Risheng Liu Pan Mu Xiaoming Yuan Shangzhi Zeng Jin Zhang AI4CE 128 36 0 16 Feb 2021
The structure of conservative gradient fields A. Lewis Tonghua Tian AI4CE 60 8 0 03 Jan 2021
Differentiable Programming à la Moreau Vincent Roulet Zaïd Harchaoui 81 5 0 31 Dec 2020
Incremental Without Replacement Sampling in Nonconvex Optimization Edouard Pauwels 61 5 0 15 Jul 2020
On Correctness of Automatic Differentiation for Non-Differentiable Functions Wonyeol Lee Hangyeol Yu Xavier Rival Hongseok Yang 83 41 0 12 Jun 2020
An Inertial Newton Algorithm for Deep Learning Camille Castera Jérôme Bolte Cédric Févotte Edouard Pauwels PINN ODL 117 64 0 29 May 2019