AdapterDrop: On the Efficiency of Adapters in Transformers

22 October 2020

Papers citing "AdapterDrop: On the Efficiency of Adapters in Transformers"

50 / 54 papers shown

Title
RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models Fujun Zhang Xiangdong Su 29 0 0 13 May 2025
Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation Xiangyu Zheng Wanyun Li Songcheng He Jianping Fan Xiaoqiang Li We Zhang VOS 25 0 0 08 Apr 2025
SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation Yixian Shen Qi Bi Jia-Hong Huang Hongyi Zhu Andy D. Pimentel Anuj Pathania 46 0 0 08 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu M. Gong 134 1 0 05 Feb 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang H. Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 104 6 0 28 Jan 2025
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 91 1 0 08 Dec 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 29 0 0 05 Oct 2024
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models Philipp Mondorf Sondre Wold Barbara Plank 29 0 0 02 Oct 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu MQ 40 0 0 22 Jul 2024
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 38 1 0 13 Jul 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris S. Jyothi 27 3 0 16 Jun 2024
Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting Divij Gupta Anubhav Bhatti Surajsinh Parmar Chen Dan Yuwei Liu Bingjie Shen San Lee AI4TS 35 2 0 16 May 2024
The Trade-off between Performance, Efficiency, and Fairness in Adapter Modules for Text Classification Minh Duc Bui K. Wense 21 0 0 03 May 2024
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees William Fleshman Aleem Khan Marc Marone Benjamin Van Durme CLL KELM 44 3 0 12 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 68 0 03 Apr 2024
Capturing Pertinent Symbolic Features for Enhanced Content-Based Misinformation Detection Flavio Merenda José Manuél Gómez-Pérez 17 0 0 29 Jan 2024
TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classification Kaipeng Zheng Weiran Huang Lichao Sun LM&MA MedIm VLM 25 0 0 12 Dec 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 16 41 0 16 Nov 2023
Scalable Neural Network Kernels Arijit Sehanobish Krzysztof Choromanski Yunfan Zhao Kumar Avinava Dubey Valerii Likhosherstov 27 4 0 20 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 13 6 0 16 Oct 2023
Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies Boshko Koloski Blaž Škrlj Marko Robnik-Šikonja Senja Pollak CLL 19 2 0 12 Sep 2023
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao-quan Song Chiwun Yang 28 29 0 23 Aug 2023
Low-Parameter Federated Learning with Large Language Models Jing Jiang Xiangyang Liu Chenyou Fan 16 24 0 26 Jul 2023
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models Shengding Hu Ning Ding Weilin Zhao Xingtai Lv Zhen Zhang Zhiyuan Liu Maosong Sun 22 7 0 05 Jul 2023
Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions Dongshuo Yin Xueting Han Bin Li Hao Feng Jinghua Bai VPVLM 26 16 0 16 Jun 2023
Adaptive Fake Audio Detection with Low-Rank Model Squeezing Xiaohui Zhang Jiangyan Yi J. Tao Chenlong Wang Le Xu Ruibo Fu 23 6 0 08 Jun 2023
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning Zhen-Ru Zhang Chuanqi Tan Haiyang Xu Chengyu Wang Jun Huang Songfang Huang 17 29 0 24 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 18 10 0 12 May 2023
Parameter-Efficient Sparse Retrievers and Rerankers using Adapters Vaishali Pal Carlos Lassance Hervé Déjean S. Clinchant 94 3 0 23 Mar 2023
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 27 81 0 19 Dec 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers A. Deshpande Md Arafat Sultan Anthony Ferritto A. Kalyan Karthik Narasimhan Avirup Sil MoE 25 1 0 29 Nov 2022
Scaling Native Language Identification with Transformer Adapters Ahmet Uluslu Gerold Schneider 11 6 0 18 Nov 2022
Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5 Irina Bigoulaeva Rachneet Sachdeva Harish Tayyar Madabushi Aline Villavicencio Iryna Gurevych LRM 20 5 0 31 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 26 27 0 20 Oct 2022
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking Tim Baumgärtner Leonardo F. R. Ribeiro Nils Reimers Iryna Gurevych 22 6 0 19 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 23 16 0 18 Oct 2022
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding Erica K. Shimomoto Edison Marrese-Taylor Hiroya Takamura Ichiro Kobayashi Hideki Nakayama Yusuke Miyao 16 7 0 26 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 120 100 0 24 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 35 138 0 12 May 2022
Adaptable Adapters N. Moosavi Quentin Delfosse Kristian Kersting Iryna Gurevych 37 20 0 03 May 2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks Chin-Lun Fu Zih-Ching Chen Yun-Ru Lee Hung-yi Lee 20 44 0 30 Apr 2022
Parameter-Efficient Abstractive Question Answering over Tables or Text Vaishali Pal Evangelos Kanoulas Maarten de Rijke LMTD 14 14 0 07 Apr 2022
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval Robert Litschko Ivan Vulić Goran Glavavs LRM 19 13 0 05 Apr 2022
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang X. Wang 20 81 0 29 Mar 2022
Continual Sequence Generation with Adaptive Compositional Modules Yanzhe Zhang Xuezhi Wang Diyi Yang KELM CLL 30 40 0 20 Mar 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 13 196 0 14 Mar 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
xGQA: Cross-Lingual Visual Question Answering Jonas Pfeiffer Gregor Geigle Aishwarya Kamath Jan-Martin O. Steitz Stefan Roth Ivan Vulić Iryna Gurevych 21 56 0 13 Sep 2021