QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 298 papers shown

Title
On the Loss of Context-awareness in General Instruction Fine-tuning Yihan Wang Andrew Bai Nanyun Peng Cho-Jui Hsieh 70 1 0 05 Nov 2024
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 77 2 0 01 Nov 2024
Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments M. Domnich Julius Valja Rasmus Moorits Veski Giacomo Magnifico Kadi Tulver Eduard Barbu Raul Vicente LRM ELM 40 2 0 28 Oct 2024
EoRA: Training-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation Shih-yang Liu Huck Yang Nai Chit Fung Nai Chit Fung Hongxu Yin ... Jan Kautz Yu-Chun Wang Pavlo Molchanov Min-Hung Chen Min-Hung Chen MQ 29 0 0 28 Oct 2024
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities Chung-En Sun Xiaodong Liu Weiwei Yang Tsui-Wei Weng Hao Cheng Aidan San Michel Galley Jianfeng Gao 39 2 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 46 4 0 23 Oct 2024
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 48 2 0 23 Oct 2024
CompAct: Compressed Activations for Memory-Efficient LLM Training Yara Shamshoum Nitzan Hodos Yuval Sieradzki Assaf Schuster MQ VLM 36 0 0 20 Oct 2024
LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content Mohamed Bayan Kmainasi Ali Ezzat Shahroor Maram Hasanain Sahinur Rahman Laskar Naeemul Hassan Firoj Alam 31 1 0 20 Oct 2024
Grammatical Error Correction for Low-Resource Languages: The Case of Zarma Mamadou K. Keita Christopher Homan Sofiane Abdoulaye Hamani Adwoa Bremang Marcos Zampieri Habibatou Abdoulaye Alfari Elysabhete Amadou Ibrahim 23 0 0 20 Oct 2024
Teaching Models to Balance Resisting and Accepting Persuasion Elias Stengel-Eskin Peter Hase Mohit Bansal MU 29 4 0 18 Oct 2024
GeoCoder: Solving Geometry Problems by Generating Modular Code through Vision-Language Models Aditya Sharma Aman Dalmia Mehran Kazemi Amal Zouaq Christopher J. Pal LRM 26 0 0 17 Oct 2024
LLM-Human Pipeline for Cultural Context Grounding of Conversations Rajkumar Pujari Dan Goldwasser 18 1 0 17 Oct 2024
Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions Michael J.Q. Zhang W. Bradley Knox Eunsol Choi 48 3 0 17 Oct 2024
DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMs Yingsong Luo Ling Chen MQ 16 0 0 16 Oct 2024
MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation S. Gorti Ilan Gofman Zhaoyan Liu Jiapeng Wu Noël Vouitsis Guangwei Yu Jesse C. Cresswell Rasa Hosseinzadeh SyDa 41 6 0 16 Oct 2024
Scaling laws for post-training quantized large language models Zifei Xu Alexander Lan W. Yazar T. Webb Sayeh Sharify Xin Eric Wang MQ 26 0 0 15 Oct 2024
Sequential LLM Framework for Fashion Recommendation Han Liu Xianfeng Tang Tianlang Chen Jiapeng Liu Indu Indu ... Roberto Fernandez Galan Michael D Porter Dongmei Jia Ning Zhang Lian Xiong AI4TS 13 1 0 15 Oct 2024
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 54 1 0 13 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 66 1 0 11 Oct 2024
Disease Entity Recognition and Normalization is Improved with Large Language Model Derived Synthetic Normalized Mentions Kuleen Sasse Shinjitha Vadlakonda Richard Kennedy J. D. Osborne AI4CE 22 0 0 10 Oct 2024
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao B. Wang Weipeng Chen Ji-Rong Wen 60 0 0 10 Oct 2024
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe Yuxin Xiao Shujian Zhang Wenxuan Zhou Marzyeh Ghassemi Sanqiang Zhao 52 0 0 07 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 55 6 0 06 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 29 0 0 05 Oct 2024
ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li X. Yan Tianao Zhang Haotong Qin Dong Xie Jiang Tian Zhongchao Shi Linghe Kong Yulun Zhang Xiaokang Yang MQ 29 2 0 04 Oct 2024
Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization Zichen Miao Zhengyuan Yang Kevin Lin Ze Wang Zicheng Liu Lijuan Wang Qiang Qiu 40 3 0 04 Oct 2024
TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models Zefang Liu Yinzhu Quan 21 0 0 02 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 48 2 0 02 Oct 2024
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging Yiming Ju Ziyi Ni Xingrun Xing Zhixiong Zeng hanyu Zhao Siqi Fan Zheng Zhang MoMe 24 2 0 01 Oct 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 66 2 0 25 Sep 2024
EventHallusion: Diagnosing Event Hallucinations in Video LLMs Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Zhiyu Tan Hao Li Jingjing Chen MLLM 59 17 0 25 Sep 2024
PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs Jiahao Yu Yangguang Shao Hanwen Miao Junzheng Shi SILM AAML 67 4 0 23 Sep 2024
Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models Haoran Ye Yuhang Xie Yuanyi Ren Hanjun Fang Xin Zhang Guojie Song LM&MA 30 1 0 18 Sep 2024
Prompt Obfuscation for Large Language Models David Pape Thorsten Eisenhofer Thorsten Eisenhofer Lea Schönherr AAML 33 2 0 17 Sep 2024
Leveraging Open-Source Large Language Models for Native Language Identification Yee Man Ng Ilia Markov 25 0 0 15 Sep 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 30 1 0 12 Sep 2024
DiVA-DocRE: A Discriminative and Voice-Aware Paradigm for Document-Level Relation Extraction Yiheng Wu Roman Yangarber Xian Mao 23 0 0 07 Sep 2024
Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models Dipankar Srirag Aditya Joshi Jacob Eisenstein 44 1 0 31 Aug 2024
Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage Md. Rafi Ur Rashid Jing Liu T. Koike-Akino Shagufta Mehnaz Ye Wang MU SILM 31 3 0 30 Aug 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 53 1 0 28 Aug 2024
Personality Alignment of Large Language Models Minjun Zhu Linyi Yang Yue Zhang Yue Zhang ALM 55 5 0 21 Aug 2024
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs Jiancheng Dong Lei Jiang Wei Jin Lu Cheng 36 1 0 18 Aug 2024
Enhancing Large Language Model-based Speech Recognition by Contextualization for Rare and Ambiguous Words Kento Nozawa Takashi Masuko Toru Taniguchi 38 1 0 15 Aug 2024
HateSieve: A Contrastive Learning Framework for Detecting and Segmenting Hateful Content in Multimodal Memes Xuanyu Su Yansong Li Diana Inkpen Nathalie Japkowicz VLM 81 2 0 11 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 38 8 0 10 Aug 2024
Learning to Rewrite: Generalized LLM-Generated Text Detection Wei Hao Ran Li Weiliang Zhao Junfeng Yang Chengzhi Mao DeLMO 31 3 0 08 Aug 2024
Conditioning LLMs with Emotion in Neural Machine Translation Charles Brazier Jean-Luc Rouas CVBM 28 2 0 06 Aug 2024
Examining Gender and Power on Wikipedia Through Face and Politeness Adil Soubki Shyne Choi Owen Rambow CVBM 28 0 0 05 Aug 2024