Batched Low-Rank Adaptation of Foundation Models

9 December 2023

Swarat Chaudhuri

Papers citing "Batched Low-Rank Adaptation of Foundation Models"

25 / 25 papers shown

Title
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 60 0 0 31 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Haozhao Wang Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 112 1 0 18 Dec 2024
Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models Yeming Wen Swarat Chaudhuri 29 0 0 11 Nov 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 29 0 0 05 Oct 2024
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint? Xi Chen Kaituo Feng Changsheng Li Xunhao Lai Xiangyu Yue Ye Yuan Guoren Wang 39 7 0 02 Oct 2024
LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models Haolin Li Yuhang Zhou Ziheng Zhao Siyuan Du Jiangchao Yao Weidi Xie Ya Zhang Yanfeng Wang 29 1 0 29 Sep 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 81 2 0 25 Sep 2024
NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers Nohil Park Heeseung Kim Che Hyun Lee Jooyoung Choi Jiheum Yeom Sungroh Yoon 23 2 0 24 Sep 2024
LoRA-Pro: Are Low-Rank Adapters Properly Optimized? Zhengbo Wang Jian Liang Ran He Zilei Wang Tieniu Tan 50 15 0 25 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 52 23 0 08 Jul 2024
Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning Ziyu Zhao Leilei Gan Guoyin Wang Yuwei Hu Tao Shen Hongxia Yang Kun Kuang Fei Wu MoE MoMe 32 11 0 24 Jun 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 28 5 0 17 Jun 2024
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors Vijay Lingam Atula Tejaswi Aditya Vavre Aneesh Shetty Gautham Krishna Gudur Joydeep Ghosh Alexandros G. Dimakis Eunsol Choi Aleksandar Bojchevski Sujay Sanghavi 40 8 0 30 May 2024
Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation Abhinav C. P. Jain Swarat Chaudhuri Thomas W. Reps Christopher M. Jermaine 18 1 0 24 May 2024
Low-Rank Knowledge Decomposition for Medical Foundation Models Yuhang Zhou Haolin Li Siyuan Du Jiangchao Yao Ya-Qin Zhang Yanfeng Wang 21 3 0 26 Apr 2024
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild Ziyu Zhao Leilei Gan Guoyin Wang Wangchunshu Zhou Hongxia Yang Kun Kuang Fei Wu MoMe 21 28 0 15 Feb 2024
The Philosopher's Stone: Trojaning Plugins of Large Language Models Tian Dong Minhui Xue Guoxing Chen Rayne Holland Shaofeng Li Yan Meng Zhen Liu Haojin Zhu AAML 13 9 0 01 Dec 2023
From Sparse to Soft Mixtures of Experts J. Puigcerver C. Riquelme Basil Mustafa N. Houlsby MoE 121 114 0 02 Aug 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 137 277 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 804 0 14 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,835 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,913 0 31 Dec 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,584 0 21 Jan 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 268 5,652 0 05 Dec 2016