ResearchTrend.AI
FixNorm: Dissecting Weight Decay for Training Deep Neural Networks

FixNorm: Dissecting Weight Decay for Training Deep Neural Networks

Yucong Zhou
Yunxiao Sun
Zhaobai Zhong
ArXivPDFHTML

Papers citing "FixNorm: Dissecting Weight Decay for Training Deep Neural Networks"

3 / 3 papers shown
Title
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Sid Black
Stella Biderman
Eric Hallahan
Quentin G. Anthony
Leo Gao
...
Shivanshu Purohit
Laria Reynolds
J. Tow
Benqi Wang
Samuel Weinbach
102
804
0
14 Apr 2022
Bag of Tricks for Image Classification with Convolutional Neural Networks
Bag of Tricks for Image Classification with Convolutional Neural Networks
Tong He
Zhi-Li Zhang
Hang Zhang
Zhongyue Zhang
Junyuan Xie
Mu Li
224
1,403
0
04 Dec 2018
A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
L. Smith
210
1,020
0
26 Mar 2018