Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

8 June 2021

Rabeeh Karimi Mahabadi

Papers citing "Compacter: Efficient Low-Rank Hypercomplex Adapter Layers"

50 / 59 papers shown

Title
GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model Zixiang Ai Zichen Liu Yuanhang Lei Zhenyu Cui Xu Zou Jiahuan Zhou 27 0 0 07 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 32 1 0 05 May 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 75 2 0 24 Feb 2025
Music for All: Representational Bias and Cross-Cultural Adaptability of Music Generation Models Atharva Mehta Shivam Chauhan Amirbek Djanibekov Atharva Kulkarni Gus Xia Monojit Choudhury 59 0 0 11 Feb 2025
DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations Krishna Sri Ipsit Mantri Carola-Bibiane Schönlieb Bruno Ribeiro Chaim Baskin Moshe Eliasof 41 0 0 09 Feb 2025
MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension Ting Liu Zunnan Xu Yue Hu Liangtao Shi Zhiqiang Wang Quanjun Yin 57 2 0 03 Jan 2025
GaLore $+$ : Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection Xutao Liao Shaohui Li Yuhui Xu Zhi Li Y. Liu You He VLM 54 2 0 31 Dec 2024
Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference Siyuan Wang Dianyi Wang Chengxing Zhou Zejun Li Zhihao Fan Xuanjing Huang Zhongyu Wei VLM 108 0 0 17 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 91 1 0 08 Dec 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 69 2 0 06 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches Rana Muhammad Shahroz Khan Pingzhi Li Sukwon Yun Zhenyu Wang S. Nirjon Chau-Wai Wong Tianlong Chen KELM 35 2 0 08 Oct 2024
Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models Hongbin Liu Lun Wang Om Thakkar Abhradeep Thakurta Arun Narayanan 21 0 0 02 Oct 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 57 6 0 10 Jul 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 56 12 0 13 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou S. Song Jianlong Wu Liqiang Nie Bernard Ghanem 45 6 0 07 Jun 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 27 2 0 05 Apr 2024
Large Convolutional Model Tuning via Filter Subspace Wei Chen Zichen Miao Qiang Qiu 49 3 0 01 Mar 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 19 1 0 18 Dec 2023
Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach Xingshuai Huang Di Wu Benoit Boulet OffRL 12 2 0 12 Dec 2023
Matrix Compression via Randomized Low Rank and Low Precision Factorization R. Saha Varun Srivastava Mert Pilanci 13 18 0 17 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 13 6 0 16 Oct 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 38 1 0 26 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 28 32 0 23 Aug 2023
Towards Instance-adaptive Inference for Federated Learning Chunhui Feng Kai Yu Nian Liu Xinxing Xu Salman Khan W. Zuo FedML 16 11 0 11 Aug 2023
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models Shengding Hu Ning Ding Weilin Zhao Xingtai Lv Zhen Zhang Zhiyuan Liu Maosong Sun 22 7 0 05 Jul 2023
Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomy I. Ciucă Y. Ting 丁 Sandor Kruk K. Iyer 6 7 0 20 Jun 2023
Group Orthogonalization Regularization For Vision Models Adaptation and Robustness Yoav Kurtz Noga Bar Raja Giryes 24 0 0 16 Jun 2023
Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions Dongshuo Yin Xueting Han Bin Li Hao Feng Jinghua Bai VPVLM 26 16 0 16 Jun 2023
Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformation Jinzi Qi Hugo Van hamme 38 3 0 12 Jun 2023
Parameter-Efficient Fine-Tuning without Introducing New Latency Baohao Liao Yan Meng Christof Monz 13 48 0 26 May 2023
Automated Few-shot Classification with Instruction-Finetuned Language Models Rami Aly Xingjian Shi Kaixiang Lin Aston Zhang A. Wilson 19 9 0 21 May 2023
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings Shibo Hao Tianyang Liu Zhen Wang Zhiting Hu RALM LLMAG 35 171 0 19 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 18 10 0 12 May 2023
Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation Julio Silva-Rodríguez Jose Dolz Ismail Ben Ayed 57 12 0 29 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 23 736 0 28 Mar 2023
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 38 35 0 17 Nov 2022
Evaluating Parameter Efficient Learning for Generation Peng-Tao Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Wei Ping Nayeon Lee M. Shoeybi Bryan Catanzaro MoE 18 3 0 25 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 15 16 0 18 Oct 2022
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models S. Kwon Jeonghoon Kim Jeongin Bae Kang Min Yoo Jin-Hwa Kim Baeseong Park Byeongwook Kim Jung-Woo Ha Nako Sung Dongsoo Lee MQ 21 30 0 08 Oct 2022
Differentially Private Optimization on Large Model at Small Cost Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 19 52 0 30 Sep 2022
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding Erica K. Shimomoto Edison Marrese-Taylor Hiroya Takamura Ichiro Kobayashi Hideki Nakayama Yusuke Miyao 10 7 0 26 Sep 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 120 98 0 24 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 35 138 0 12 May 2022
Hypercomplex Image-to-Image Translation Eleonora Grassucci Luigi Sigillo A. Uncini Danilo Comminiello 16 7 0 04 May 2022
IDPG: An Instance-Dependent Prompt Generation Method Zhuofeng Wu Sinong Wang Jiatao Gu Rui Hou Yuxiao Dong V. Vydiswaran Hao Ma VLM 19 58 0 09 Apr 2022
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang X. Wang 20 80 0 29 Mar 2022
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 23 96 0 25 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 344 0 13 Oct 2021