Distillation Scaling Laws

12 February 2025

Papers citing "Distillation Scaling Laws"

7 / 7 papers shown

Title
EvoLM: In Search of Lost Language Model Training Dynamics Zhenting Qi Fan Nie Alexandre Alahi James Zou Himabindu Lakkaraju Yilun Du Eric P. Xing Sham Kakade Hanlin Zhang 31 1 0 19 Jun 2025
Improved Scaling Laws in Linear Regression via Data Reuse Licong Lin Jingfeng Wu Peter Bartlett 24 0 0 10 Jun 2025
SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought Guanghao Li Wenhao Jiang Mingfeng Chen Yan Li Hao Yu Shuting Dong Tao Ren Ming Tang Chun Yuan ReLM LRM 28 0 0 30 May 2025
Improving Respiratory Sound Classification with Architecture-Agnostic Knowledge Distillation from Ensembles Miika Toikkanen June-Woo Kim 45 0 0 28 May 2025
Scalable Strategies for Continual Learning with Replay Truman Hickok CLL 87 0 0 18 May 2025
Scaling Laws for Data-Efficient Visual Transfer Learning Wenxuan Yang Qingqu Wei Chenxi Ma Weimin Tan Bo Yan 49 1 0 17 Apr 2025
Scaling Laws of Synthetic Data for Language Models Zeyu Qin Qingxiu Dong Xingxing Zhang Li Dong Xiaolong Huang ... Hany Awadalla Yi R. Fung Weizhu Chen Minhao Cheng Furu Wei SyDa 139 7 0 25 Mar 2025