Selecting Large Language Model to Fine-tune via Rectified Scaling Law

Selecting Large Language Model to Fine-tune via Rectified Scaling Law

4 February 2024

Sujian Li

Xiaojun Wan

Papers citing "Selecting Large Language Model to Fine-tune via Rectified Scaling Law"

12 / 12 papers shown

Title
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection Xinyue Zeng Haohui Wang Junhong Lin Jun Wu Tyler Cody Dawei Zhou 31 0 0 01 May 2025
Compute-Constrained Data Selection Junjie Oscar Yin Alexander M. Rush 33 0 0 21 Oct 2024
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 41 7 0 11 Oct 2024
Scaling Optimal LR Across Token Horizons Johan Bjorck Alon Benhaim Vishrav Chaudhary Furu Wei Xia Song 35 4 0 30 Sep 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 125 298 0 05 Jan 2024
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions Minghao Wu Abdul Waheed Chiyu Zhang Muhammad Abdul-Mageed Alham Fikri Aji ALM 115 115 0 27 Apr 2023
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 145 101 0 13 Sep 2022
Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance Shibal Ibrahim Natalia Ponomareva Rahul Mazumder AAML 97 15 0 13 Oct 2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 181 89 0 22 Sep 2021
A linearized framework and a new benchmark for model selection for fine-tuning Aditya Deshpande Alessandro Achille Avinash Ravichandran Hao Li L. Zancato Charless C. Fowlkes Rahul Bhotika Stefano Soatto Pietro Perona ALM 103 37 0 29 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020
Transferability and Hardness of Supervised Classification Tasks Anh Tran Cuong V Nguyen Tal Hassner 134 163 0 21 Aug 2019