Towards Optimal Learning of Language Models

Towards Optimal Learning of Language Models

27 February 2024

Qingxiu Dong

Papers citing "Towards Optimal Learning of Language Models"

5 / 5 papers shown

Title
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 54 5 0 22 Oct 2024
Data Selection via Optimal Control for Language Models Yuxian Gu Li Dong Hongning Wang Y. Hao Qingxiu Dong Furu Wei Minlie Huang AI4CE 38 4 0 09 Oct 2024
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance Jiasheng Ye Peiju Liu Tianxiang Sun Yunhua Zhou Jun Zhan Xipeng Qiu 35 58 0 25 Mar 2024
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 41 44 0 14 Jul 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020