QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

48 / 298 papers shown

Title
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 32 30 0 11 Dec 2023
Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey Haotian Zhang S. D. Semujju Zhicheng Wang Xianwei Lv Kang Xu ... Jing Wu Zhuo Long Wensheng Liang Xiaoguang Ma Ruiyan Zhuang UQCV AI4TS AI4CE 27 4 0 11 Dec 2023
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 20 15 0 07 Dec 2023
Run LoRA Run: Faster and Lighter LoRA Implementations Daria Cherniuk A. Mikhalev Ivan V. Oseledets AI4CE 8 1 0 06 Dec 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 32 11 0 23 Nov 2023
Inducing Political Bias Allows Language Models Anticipate Partisan Reactions to Controversies Zihao He Siyi Guo Ashwin Rao Kristina Lerman 16 2 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 16 41 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 19 27 0 15 Nov 2023
Interaction is all You Need? A Study of Robots Ability to Understand and Execute Kushal Koshti Nidhir Bhavsar 45 1 0 13 Nov 2023
BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddings Xianming Li Jing Li 34 10 0 09 Nov 2023
Chain of Images for Intuitively Reasoning Fanxu Meng Haotong Yang Yiding Wang Muhan Zhang LRM 28 6 0 09 Nov 2023
LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detection Ahmad Nasir Aadish Sharma Kokil Jaidka Saifuddin Ahmed 32 3 0 29 Oct 2023
Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing Xinyu Hu Pengfei Tang Simiao Zuo Zihan Wang Bowen Song Qiang Lou Jian Jiao Denis Xavier Charles LRM 33 7 0 20 Oct 2023
Explainable Depression Symptom Detection in Social Media Eliseo Bao Souto Anxo Perez Javier Parapar 22 5 0 20 Oct 2023
Audio Editing with Non-Rigid Text Prompts Francesco Paissan Luca Della Libera Zhepei Wang Mirco Ravanelli Paris Smaragdis Cem Subakan DiffM 24 5 0 19 Oct 2023
JMedLoRA:Medical Domain Adaptation on Japanese Large Language Models using Instruction-tuning Issey Sukeda Masahiro Suzuki Hiroki Sakaji Satoshi Kodera LM&MA 25 7 0 16 Oct 2023
ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models Haoran Luo E. Haihong Zichen Tang Shiyao Peng Yikai Guo ... Guanting Dong Meina Song Wei Lin Yifan Zhu Luu Anh Tuan RALM 26 37 0 13 Oct 2023
LLaMA Rider: Spurring Large Language Models to Explore the Open World Yicheng Feng Yuxuan Wang Jiazheng Liu Sipeng Zheng Zongqing Lu LLMAG LRM 11 15 0 13 Oct 2023
MuseChat: A Conversational Music Recommendation System for Videos Zhikang Dong Bin Chen Xiulong Liu Paweł Polak Peng Zhang LRM 34 25 0 10 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 12 2 0 08 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 24 92 0 27 Sep 2023
CPLLM: Clinical Prediction with Large Language Models Ofir Ben Shoham Nadav Rappoport LM&MA 19 24 0 20 Sep 2023
OpenChat: Advancing Open-source Language Models with Mixed-Quality Data Guan-Bo Wang Sijie Cheng Xianyuan Zhan Xiangang Li Sen Song Yang Liu ALM 8 227 0 20 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 31 32 0 23 Aug 2023
In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning Xiaochuang Han 17 19 0 08 Aug 2023
SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool Youyang Ng Daisuke Miyashita Yasuto Hoshi Yasuhiro Morioka Osamu Torii Tomoya Kodama J. Deguchi RALM 8 9 0 08 Aug 2023
From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application? Rodrigo Pedro Daniel Castro Paulo Carreira Nuno Santos SILM AAML 31 50 0 03 Aug 2023
Scaling Sentence Embeddings with Large Language Models Ting Jiang Shaohan Huang Zhongzhi Luan Deqing Wang Fuzhen Zhuang LRM 34 40 0 31 Jul 2023
Three Bricks to Consolidate Watermarks for Large Language Models Pierre Fernandez Antoine Chaffin Karim Tit Vivien Chappelier Teddy Furon WaLM 9 46 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 18 116 0 25 Jul 2023
Mini-Giants: "Small" Language Models and Open Source Win-Win Zhengping Zhou Lezhi Li Xinxi Chen Andy Li SyDa ALM MoE 24 5 0 17 Jul 2023
My3DGen: A Scalable Personalized 3D Generative Model Luchao Qi Jiaye Wu Annie N. Wang Sheng-Yu Wang Roni Sengupta 3DH 30 3 0 11 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 20 40 0 03 Jul 2023
SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking Chris Cundy Stefano Ermon 11 10 0 08 Jun 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services Yansong Li Zhixing Tan Yang Liu SILM VLM 43 63 0 10 May 2023
Multi-step Jailbreaking Privacy Attacks on ChatGPT Haoran Li Dadi Guo Wei Fan Mingshi Xu Jie Huang Fanpu Meng Yangqiu Song SILM 34 317 0 11 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 576 0 06 Apr 2023
Commentary Generation from Data Records of Multiplayer Strategy Esports Game Zihan Wang Naoki Yoshinaga 16 0 0 21 Dec 2022
Controllable Citation Sentence Generation with Language Models Nianlong Gu Richard H. R. Hahnloser 25 2 0 14 Nov 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 242 1,070 0 05 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 225 495 0 28 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 335 0 02 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 205 1,651 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Hypothesis Only Baselines in Natural Language Inference Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme 187 574 0 02 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018