Retrieval-based Knowledge Transfer: An Effective Approach for Extreme
Large Language Model Compression

Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression

24 October 2023

Dongyan Zhao

Rui Yan

Papers citing "Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression"

9 / 9 papers shown

Title
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Janet Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Y. Zhang ALM 53 0 0 24 Apr 2025
ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT S. Ubani S. Polat Rodney D. Nielsen 84 51 0 27 Apr 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 240 1,070 0 05 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Towards Zero-Label Language Learning Zirui Wang Adams Wei Yu Orhan Firat Yuan Cao SyDa 167 101 0 19 Sep 2021
Distilling Linguistic Context for Language Model Compression Geondo Park Gyeongman Kim Eunho Yang 32 30 0 17 Sep 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 273 1,561 0 18 Sep 2019
Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback Carolin (Haas) Lawrence Stefan Riezler OffRL 169 56 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018