Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2306.09782
Cited By
Full Parameter Fine-tuning for Large Language Models with Limited Resources
16 June 2023
Kai Lv
Yuqing Yang
Tengxiao Liu
Qi-jie Gao
Qipeng Guo
Xipeng Qiu
Re-assign community
ArXiv
PDF
HTML
Papers citing
"Full Parameter Fine-tuning for Large Language Models with Limited Resources"
28 / 78 papers shown
Title
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Jiawei Zhao
Zhenyu (Allen) Zhang
Beidi Chen
Zhangyang Wang
A. Anandkumar
Yuandong Tian
41
173
0
06 Mar 2024
A Survey of Large Language Models in Cybersecurity
Gabriel de Jesus Coelho da Silva
Carlos Becker Westphall
27
5
0
26 Feb 2024
Why Transformers Need Adam: A Hessian Perspective
Yushun Zhang
Congliang Chen
Tian Ding
Ziniu Li
Ruoyu Sun
Zhimin Luo
24
40
0
26 Feb 2024
Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks
Minju Seo
Jinheon Baek
James Thorne
Sung Ju Hwang
RALM
29
9
0
21 Feb 2024
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Shuai Zhao
Leilei Gan
Anh Tuan Luu
Jie Fu
Lingjuan Lyu
Meihuizi Jia
Jinming Wen
AAML
21
22
0
19 Feb 2024
Higher Layers Need More LoRA Experts
Chongyang Gao
Kezhen Chen
Jinmeng Rao
Baochen Sun
Ruibo Liu
Daiyi Peng
Yawen Zhang
Xiaoyuan Guo
Jie Yang
V. Subrahmanian
MoE
13
37
0
13 Feb 2024
InCoRo: In-Context Learning for Robotics Control with Feedback Loops
Jiaqiang Ye Zhu
Carla Gomez Cano
David Vazquez Bermudez
Michal Drozdzal
LRM
22
7
0
07 Feb 2024
Flora: Low-Rank Adapters Are Secretly Gradient Compressors
Yongchang Hao
Yanshuai Cao
Lili Mou
6
39
0
05 Feb 2024
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy
Yongkang Liu
Yiqun Zhang
Qian Li
Tong Liu
Shi Feng
Daling Wang
Yifei Zhang
Hinrich Schütze
33
6
0
26 Jan 2024
IoT in the Era of Generative AI: Vision and Challenges
Xin Wang
Zhongwei Wan
Arvin Hekmati
M. Zong
Samiul Alam
Mi Zhang
Bhaskar Krishnamachari
27
15
0
03 Jan 2024
RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
André Silva
Sen Fang
Martin Monperrus
MoMe
KELM
62
37
0
25 Dec 2023
ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-order Optimization
Shuoran Jiang
Qingcai Chen
Youcheng Pan
Yang Xiang
Yukang Lin
Xiangping Wu
Chuanyi Liu
Xiaobao Song
ODL
28
9
0
23 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models
Weixi Song
Z. Li
Lefei Zhang
Hai Zhao
Bo Du
VLM
16
6
0
19 Dec 2023
ASPEN: High-Throughput LoRA Fine-Tuning of Large Language Models with a Single GPU
Zhengmao Ye
Dengchun Li
Jingqi Tian
Tingfeng Lan
Jie Zuo
...
Hui Lu
Yexi Jiang
Jian Sha
Ke Zhang
Mingjie Tang
91
7
0
05 Dec 2023
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
Kai Lv
Shuo Zhang
Tianle Gu
Shuhao Xing
Jiawei Hong
...
Tengxiao Liu
Yu Sun
Penousal Machado
Hang Yan
Xipeng Qiu
35
7
0
01 Dec 2023
Cerbero-7B: A Leap Forward in Language-Specific LLMs Through Enhanced Chat Corpus Generation and Evaluation
Federico A. Galatolo
M. G. Cimino
24
5
0
27 Nov 2023
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Ziniu Li
Tian Xu
Yushun Zhang
Zhihang Lin
Yang Yu
Ruoyu Sun
Zhimin Luo
19
47
0
16 Oct 2023
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Kai Lv
Hang Yan
Qipeng Guo
Haijun Lv
Xipeng Qiu
ODL
19
20
0
16 Oct 2023
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
Zhikai Li
Xiaoxuan Liu
Banghua Zhu
Zhen Dong
Qingyi Gu
Kurt Keutzer
MQ
27
7
0
11 Oct 2023
FireAct: Toward Language Agent Fine-tuning
Baian Chen
Chang Shu
Ehsan Shareghi
Nigel Collier
Karthik Narasimhan
Shunyu Yao
ALM
LLMAG
99
97
0
09 Oct 2023
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices
Bojia Zi
Xianbiao Qi
Lingzhi Wang
Jianan Wang
Kam-Fai Wong
Lei Zhang
20
40
0
05 Sep 2023
Instruction Tuning for Large Language Models: A Survey
Shengyu Zhang
Linfeng Dong
Xiaoya Li
Sen Zhang
Xiaofei Sun
...
Jiwei Li
Runyi Hu
Tianwei Zhang
Fei Wu
Guoyin Wang
LM&MA
21
531
0
21 Aug 2023
LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models
Yixuan Weng
Zhiqi Wang
Huanxuan Liao
Shizhu He
Shengping Liu
Kang Liu
Jun Zhao
26
3
0
20 Aug 2023
A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrology
Sean Wu
Michael Koo
L. Blum
A. Black
Liyo Kao
Fabien Scalzo
Ira Kurtz
LM&MA
ELM
AI4MH
18
41
0
09 Aug 2023
Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning
Deepanway Ghosal
Yew Ken Chia
Navonil Majumder
Soujanya Poria
ALM
LRM
30
17
0
05 Jul 2023
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing
Walid Hariri
AI4MH
LM&MA
25
84
0
27 Mar 2023
Diversity Over Size: On the Effect of Sample and Topic Sizes for Argument Mining Datasets
Benjamin Schiller
Johannes Daxenberger
Iryna Gurevych
26
1
0
23 May 2022
ZeRO-Offload: Democratizing Billion-Scale Model Training
Jie Ren
Samyam Rajbhandari
Reza Yazdani Aminabadi
Olatunji Ruwase
Shuangyang Yang
Minjia Zhang
Dong Li
Yuxiong He
MoE
160
413
0
18 Jan 2021
Previous
1
2