Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models

2 May 2023

Papers citing "Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models"

13 / 13 papers shown

Title
MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution Yuxuan Jiang Chen Feng Fan Zhang David Bull SupR 22 11 0 15 Apr 2024
Learning Evaluation Models from Large Language Models for Sequence Generation Chenglong Wang Hang Zhou Kai-Chun Chang Tongran Liu Chunliang Zhang Quan Du Tong Xiao Yue Zhang Jingbo Zhu ELM 31 3 0 08 Aug 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 198 283 0 03 May 2023
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts Nghia T. Le Fan Bai Alan Ritter 29 12 0 07 Oct 2022
Improving Large-scale Paraphrase Acquisition and Generation Yao Dou Chao Jiang Wei-ping Xu 42 9 0 06 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models Qinyuan Ye Madian Khabsa M. Lewis Sinong Wang Xiang Ren Aaron Jaech 19 5 0 16 Oct 2021
Clean or Annotate: How to Spend a Limited Data Collection Budget Derek Chen Zhou Yu Samuel R. Bowman 19 13 0 15 Oct 2021
Predicting Attention Sparsity in Transformers Marcos Vinícius Treviso António Góis Patrick Fernandes E. Fonseca André F. T. Martins 27 13 0 24 Sep 2021
Pre-train or Annotate? Domain Adaptation with a Constrained Budget Fan Bai Alan Ritter Wei-ping Xu 52 28 0 10 Sep 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 233 626 0 21 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 275 3,784 0 18 Apr 2021