QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 298 papers shown

Title
Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Stanislas Laborde Martin Cousseau Antoun Yaacoub Lionel Prevost MQ 18 0 0 12 May 2025
Stability in Single-Peaked Strategic Resource Selection Games Henri Zeiler 21 3 0 09 May 2025
GroverGPT-2: Simulating Grover's Algorithm via Chain-of-Thought Reasoning and Quantum-Native Tokenization Min Chen Jinglei Cheng Pingzhi Li Haoran Wang Tianlong Chen Junyu Liu LRM 34 0 0 08 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 41 0 0 08 May 2025
HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights Ozan Gokdemir Carlo Siebenschuh Alexander Brace Azton Wells Brian Hsu ... A. Anandkumar Ian Foster R. Stevens V. Vishwanath A. Ramanathan VLM 30 0 0 07 May 2025
GASCADE: Grouped Summarization of Adverse Drug Event for Enhanced Cancer Pharmacovigilance Sofia Jamil Aryan Dabad Bollampalli Areen Reddy S. Saha Rajiv Misra Adil A. Shakur 51 0 0 07 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 32 1 0 05 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 47 0 0 05 May 2025
An Automated LLM-based Pipeline for Asset-Level Database Creation to Assess Deforestation Impact Avanija Menon Ovidiu Serban 19 0 0 05 May 2025
EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices Arnab Sanyal Prithwish Mukherjee Gourav Datta Sandeep P. Chinchali MQ 50 0 0 05 May 2025
Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach Jiancong Xiao Bojian Hou Zhanliang Wang Ruochen Jin Q. Long Weijie Su Li Shen 28 0 0 04 May 2025
Exploring the Role of Diversity in Example Selection for In-Context Learning Janak Kapuriya Manit Kaushik Debasis Ganguly S. Bhatia 15 0 0 03 May 2025
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 34 0 0 03 May 2025
An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understanding Siyang Jiang Bufang Yang Lilin Xu Mu Yuan Yeerzhati Abudunuer ... Liekang Zeng Hongkai Chen Zhenyu Yan Xiaofan Jiang Guoliang Xing VLM 51 0 0 03 May 2025
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 25 0 0 02 May 2025
ICQuant: Index Coding enables Low-bit LLM Quantization Xinlin Li Osama A. Hanna Christina Fragouli Suhas Diggavi MQ 58 0 0 01 May 2025
Fast and Low-Cost Genomic Foundation Models via Outlier Removal Haozheng Luo Chenghao Qiu Maojiang Su Zhihan Zhou Zoe Mehta Guo Ye Jerry Yao-Chieh Hu Han Liu AAML 55 0 0 01 May 2025
Block Circulant Adapter for Large Language Models Xinyu Ding Meiqi Wang Siyu Liao Zhongfeng Wang 29 0 0 01 May 2025
Homa at SemEval-2025 Task 5: Aligning Librarian Records with OntoAligner for Subject Tagging Hadi Bayrami Asl Tekanlou J. Razmara Mahsa Sanaei Mostafa Rahgouy Hamed Babaei Giglou 32 0 0 30 Apr 2025
COSMOS: Predictable and Cost-Effective Adaptation of LLMs Jiayu Wang Aws Albarghouthi Frederic Sala 45 0 0 30 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 81 0 0 29 Apr 2025
BrAIcht, a theatrical agent that speaks like Bertolt Brecht's characters Baz Roland Kristina Malyseva Anna Pappa Tristan Cazenave 59 0 0 29 Apr 2025
TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts Pradip Kunwar Minh Vu Maanak Gupta Mahmoud Abdelsalam Manish Bhattarai MoE MoMe 52 0 0 29 Apr 2025
AKIBoards: A Structure-Following Multiagent System for Predicting Acute Kidney Injury David L. Gordon P. Petousis S. Nicholas Alex A. T. Bui FAtt 50 0 0 29 Apr 2025
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Z. Yang Tong Liu Qingrong Xia Xinyu Duan Z. Wang Baoxing Huai M. Zhang LLMAG 77 0 0 28 Apr 2025
Versatile Framework for Song Generation with Prompt-based Control Y. Zhang Wenxiang Guo Changhao Pan Z. Zhu Ruiqi Li ... Rongjie Huang Ruiyuan Zhang Zhiqing Hong Ziyue Jiang Zhou Zhao 71 1 0 27 Apr 2025
Platonic Grounding for Efficient Multimodal Language Models Moulik Choraria Xinbo Wu Akhil Bhimaraju Nitesh Sekhar Yue Wu Xu Zhang Prateek Singhal L. Varshney 54 0 0 27 Apr 2025
Optimizing Multi-Round Enhanced Training in Diffusion Models for Improved Preference Understanding Kun Li J. Wang Yangfan He Xinyuan Song Ruoyu Wang ... K. Li Sida Li Miao Zhang Tianyu Shi Xueqian Wang 40 0 0 25 Apr 2025
Pushing the boundary on Natural Language Inference Pablo Miralles-González Javier Huertas-Tato Alejandro Martín David Camacho LRM 39 0 0 25 Apr 2025
Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions James D. Finch Yasasvi Josyula Jinho D. Choi 38 0 0 25 Apr 2025
Synergizing RAG and Reasoning: A Systematic Review Yunfan Gao Yun Xiong Yijie Zhong Yuxi Bi Ming Xue H. Wang LRM AI4CE 40 1 0 22 Apr 2025
LLM-Driven Usefulness Judgment for Web Search Evaluation Mouly Dewan Jiqun Liu Aditya Gautam Chirag Shah 45 0 0 19 Apr 2025
Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management Hang Zhang Jiuchen Shi Yixiao Wang Quan Chen Yizhou Shan Minyi Guo 25 0 0 19 Apr 2025
LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identification Yiding Lu Mouxing Yang Dezhong Peng Peng Hu Yijie Lin Xi Peng 55 0 0 14 Apr 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 29 1 0 12 Apr 2025
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking Chris Samarinas Hamed Zamani ALM LRM 66 0 0 04 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 44 2 0 02 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 53 0 0 01 Apr 2025
Benchmarking Systematic Relational Reasoning with Large Language and Reasoning Models Irtaza Khalid Amir Masoud Nourollah Steven Schockaert LRM 38 0 0 30 Mar 2025
Towards Symmetric Low-Rank Adapters Tales Panoutsos Rodrygo L. T. Santos Flavio Figueiredo 26 0 0 29 Mar 2025
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang Chi-chih Chang N. Frumkin Kai-Chiang Wu Mohamed S. Abdelfattah Diana Marculescu MQ 66 0 0 28 Mar 2025
Explainable ICD Coding via Entity Linking Leonor Barreiros I. Coutinho Gonçalo M. Correia Bruno Martins 55 0 0 26 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 46 0 0 13 Mar 2025
Validating LLM-as-a-Judge Systems in the Absence of Gold Labels Luke M. Guerdan Solon Barocas Kenneth Holstein Hanna M. Wallach Zhiwei Steven Wu Alexandra Chouldechova ALM ELM 126 0 0 13 Mar 2025
Leveraging Retrieval Augmented Generative LLMs For Automated Metadata Description Generation to Enhance Data Catalogs Mayank Singh Abhijeet Kumar Sasidhar Donaparthi Gayatri Karambelkar 40 0 0 12 Mar 2025
Prompt Inversion Attack against Collaborative Inference of Large Language Models Wenjie Qu Yuguang Zhou Yongji Wu Tingsong Xiao Binhang Yuan Y. Li Jiaheng Zhang 66 0 0 12 Mar 2025
MoFE: Mixture of Frozen Experts Architecture Jean Seo Jaeyoon Kim Hyopil Shin MoE 77 0 0 09 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 67 0 0 04 Mar 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 33 0 0 28 Feb 2025
FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework S M Sarwar AI4MH 39 0 0 27 Feb 2025