VeRA: Vector-based Random Matrix Adaptation

17 October 2023

Papers citing "VeRA: Vector-based Random Matrix Adaptation"

50 / 103 papers shown

Title
AutoLoRA: AutoGuidance Meets Low-Rank Adaptation for Diffusion Models Artur Kasymov Marcin Sendera Michał Stypułkowski Maciej Ziȩba Przemysław Spurek 33 1 0 04 Oct 2024
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards Sheng Wang Liheng Chen Pengan Chen Jingwei Dong Boyang Xue Jiyue Jiang Lingpeng Kong Chuan Wu MoE 31 8 0 01 Oct 2024
Mining Your Own Secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models Saurav Jha Shiqi Yang Masato Ishii Mengjie Zhao Christian Simon Muhammad Jehanzeb Mirza Dong Gong Lina Yao Shusuke Takahashi Yuki Mitsufuji DiffM 67 2 0 01 Oct 2024
LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models Haolin Li Yuhang Zhou Ziheng Zhao Siyuan Du Jiangchao Yao Weidi Xie Ya Zhang Yanfeng Wang 31 1 0 29 Sep 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 172 2 0 25 Sep 2024
NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers Nohil Park Heeseung Kim Che Hyun Lee Jooyoung Choi Jiheum Yeom Sungroh Yoon 27 2 0 24 Sep 2024
HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation Geyuan Zhang Xiaofei Zhou Chuheng Chen 29 0 0 20 Sep 2024
Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models Divij Gupta Anubhav Bhatti Surajsinh Parmar AI4TS 31 1 0 17 Sep 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 48 1 0 12 Sep 2024
Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopy Bojian Li Bo Liu Jinghua Yue F. Zhou Fugen Zhou MedIm MDE 53 2 0 12 Sep 2024
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation Zhe Cao Zhi Qu Hidetaka Kamigaito Taro Watanabe MoE 35 0 0 08 Sep 2024
SARA: Singular-Value Based Adaptive Low-Rank Adaption Jihao Gu Shuai Chen Zelin Wang Yibo Zhang Ping Gong 38 3 0 06 Aug 2024
Parameter-Efficient Fine-Tuning via Circular Convolution Aochuan Chen Jiashun Cheng Zijing Liu Ziqi Gao Fugee Tsung Yu Li Jia Li 56 2 0 27 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
LoRA-PT: Low-Rank Adapting UNETR for Hippocampus Segmentation Using Principal Tensor Singular Values and Vectors Guanghua He Wangang Cheng Hancan Zhu Gaohang Yu 45 1 0 16 Jul 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 43 7 0 10 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 45 5 0 07 Jul 2024
Octo-planner: On-device Language Model for Planner-Action Agents Wei Chen Zhiyuan Li Zhen Guo Yikang Shen LM&Ro LLMAG 31 4 0 26 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 45 13 0 24 Jun 2024
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation Seyedarmin Azizi Souvik Kundu Massoud Pedram 32 7 0 18 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 48 5 0 17 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris Sangeetha Abdu Jyothi 32 3 0 16 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 49 22 0 16 Jun 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 44 19 0 13 Jun 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 36 13 0 12 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 13 0 06 Jun 2024
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need Martin Wistuba Prabhu Teja Sivaprasad Lukas Balles Giovanni Zappella 40 0 0 05 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 36 4 0 05 Jun 2024
LoFiT: Localized Fine-tuning on LLM Representations Fangcong Yin Xi Ye Greg Durrett 38 13 0 03 Jun 2024
Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models Xinxi Zhang Song Wen Ligong Han Felix Juefei Xu Akash Srivastava Junzhou Huang Hao Wang Molei Tao Dimitris N. Metaxas DiffM 36 6 0 31 May 2024
Low-rank finetuning for LLMs: A fairness perspective Saswat Das Marco Romanelli Cuong Tran Zarreen Reza B. Kailkhura Ferdinando Fioretto 40 1 0 28 May 2024
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning Pengxiang Li Lu Yin Xiaowei Gao Shiwei Liu 34 7 0 28 May 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 48 0 0 28 May 2024
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters Klaudia Bałazy Mohammadreza Banaei Karl Aberer Jacek Tabor 43 19 0 27 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 44 2 0 24 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning Ting Jiang Shaohan Huang Shengyue Luo Zihan Zhang Haizhen Huang ... Weiwei Deng Feng Sun Qi Zhang Deqing Wang Fuzhen Zhuang 37 33 0 20 May 2024
LoRA Learns Less and Forgets Less D. Biderman Jose Javier Gonzalez Ortiz Jacob P. Portes Mansheej Paul Philip Greengard ... Sam Havens Vitaliy Chiley Jonathan Frankle Cody Blakeney John P. Cunningham CLL 43 112 0 15 May 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 38 58 0 04 Apr 2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models Zequan Liu Jiawen Lyn Wei-wei Zhu Xing Tian Yvette Graham OffRL 40 10 0 24 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 318 0 21 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 46 2 0 14 Mar 2024
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning Pengjie Ren Chengshun Shi Shiguang Wu Mengqi Zhang Zhaochun Ren Maarten de Rijke Zhumin Chen Jiahuan Pei MoE 41 14 0 27 Feb 2024
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA Sheng Wang Boyang Xue Jiacheng Ye Jiyue Jiang Liheng Chen Lingpeng Kong Chuan Wu 30 14 0 24 Feb 2024
Advancing Parameter Efficiency in Fine-tuning via Representation Editing Muling Wu Wenhao Liu Xiaohua Wang Tianlong Li Changze Lv Zixuan Ling Jianhao Zhu Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 28 19 0 23 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 43 148 0 19 Feb 2024
Overview of the PromptCBLUE Shared Task in CHIP2023 Wei-wei Zhu Xiaoling Wang Mosha Chen Buzhou Tang LM&MA ELM 39 7 0 29 Dec 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 32 42 0 16 Nov 2023