Predicting the Impact of Model Expansion through the Minima Manifold: A Loss Landscape Perspective

24 May 2024

Papers citing "Predicting the Impact of Model Expansion through the Minima Manifold: A Loss Landscape Perspective"

5 / 5 papers shown

Title
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks Xiang Wang Annie Wang Mo Zhou Rong Ge MoMe 158 10 0 03 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 49 0 23 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 4,424 0 23 Jan 2020
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,290 0 05 Nov 2016