A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on
Chinese Instruction Data for Instruction Following Large Language Model

A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model

17 April 2023

Xiangang Li

Papers citing "A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model"

19 / 19 papers shown

Title
Sparsity Outperforms Low-Rank Projections in Few-Shot Adaptation Nairouz Mrabah Nicolas Richet Ismail Ben Ayed Eric Granger BDL VLM 47 0 0 16 Apr 2025
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 48 2 0 23 Oct 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 26 5 0 28 Apr 2024
SambaLingo: Teaching Large Language Models New Languages Zoltan Csaki Bo Li Jonathan Li Qiantong Xu Pian Pawakapan Leon Zhang Yun Du Hengyu Zhao Changran Hu Urmish Thakker 32 6 0 08 Apr 2024
Token-Efficient Leverage Learning in Large Language Models Yuanhao Zeng Min Wang Yihang Wang Yingxia Shao 29 0 0 01 Apr 2024
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy Yongkang Liu Yiqun Zhang Qian Li Tong Liu Shi Feng Daling Wang Yifei Zhang Hinrich Schütze 33 6 0 26 Jan 2024
Code-Based English Models Surprising Performance on Chinese QA Pair Extraction Task Linghan Zheng Hui Liu Xiaojun Lin Jiayuan Dong Yue Sheng Gang Shi Zhiwei Liu Hongwei Chen 19 0 0 16 Jan 2024
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 32 32 0 11 Dec 2023
Learning From Mistakes Makes LLM Better Reasoner Shengnan An Zexiong Ma Zeqi Lin Nanning Zheng Jian-Guang Lou Weizhu Chen LRM 19 75 0 31 Oct 2023
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning Xuansheng Wu Wenlin Yao Jianshu Chen Xiaoman Pan Xiaoyang Wang Ninghao Liu Dong Yu LRM 20 26 0 30 Sep 2023
ConPET: Continual Parameter-Efficient Tuning for Large Language Models Chenyan Song Xu Han Zheni Zeng Kuai Li Chen Chen Zhiyuan Liu Maosong Sun Taojiannan Yang CLL KELM 14 9 0 26 Sep 2023
Using fine-tuning and min lookahead beam search to improve Whisper Andrea Do Oscar Brown Zhengjie Wang Nikhil Mathew Zixin Liu Jawwad Ahmed Cheng Yu 25 1 0 19 Sep 2023
Efficient RLHF: Reducing the Memory Usage of PPO Michael Santacroce Yadong Lu Han Yu Yuan-Fang Li Yelong Shen 27 27 0 01 Sep 2023
Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals Simo Ryu S. Seo Jaejun Yoo 16 4 0 28 May 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 245 1,071 0 05 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 209 179 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,843 0 18 Apr 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,583 0 18 Sep 2019