How to guess a gradient

How to guess a gradient

7 December 2023

Utkarsh Singhal

Jonathan Ragan-Kelley

Joshua B. Tenenbaum

Papers citing "How to guess a gradient"

8 / 8 papers shown

Title
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM Yehonathan Refael Iftach Arbel Ofir Lindenbaum Tom Tirer 64 0 0 26 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 48 3 0 31 Jan 2025
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang ... Wotao Yin Mingyi Hong Zhangyang Wang Sijia Liu Tianlong Chen 20 45 0 18 Feb 2024
One Forward is Enough for Neural Network Training via Likelihood Ratio Method Jinyang Jiang Zeliang Zhang Chenliang Xu Zhaofei Yu Yijie Peng 23 11 0 15 May 2023
Scaling Forward Gradient With Local Losses Mengye Ren Simon Kornblith Renjie Liao Geoffrey E. Hinton 73 49 0 07 Oct 2022
Optimization without Backpropagation Gabriel Belouze 19 7 0 13 Sep 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,843 0 18 Apr 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020