Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 765 papers shown

Title
Decoder Tuning: Efficient Language Understanding as Decoding Ganqu Cui Wentao Li Ning Ding Longtao Huang Zhiyuan Liu Maosong Sun 19 6 0 16 Dec 2022
Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systems Denis Emelin Daniele Bonadiman Sawsan Alqahtani Yi Zhang Saab Mansour 13 17 0 15 Dec 2022
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao L. Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 21 5 0 08 Dec 2022
Continual Learning for On-Device Speech Recognition using Disentangled Conformers Anuj Diwan Ching-Feng Yeh Wei-Ning Hsu Paden Tomasello Eunsol Choi David F. Harwath Abdel-rahman Mohamed CLL BDL 25 7 0 02 Dec 2022
CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models Zih-Ching Chen Yu-Shun Sung Hung-yi Lee 23 16 0 01 Dec 2022
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors Hamish Ivison Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 31 19 0 01 Dec 2022
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Shafiq R. Joty Steven C. H. Hoi OOD VLM 18 5 0 30 Nov 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers A. Deshpande Md Arafat Sultan Anthony Ferritto A. Kalyan Karthik Narasimhan Avirup Sil MoE 33 1 0 29 Nov 2022
Few-shot Query-Focused Summarization with Prefix-Merging Ruifeng Yuan Zili Wang Ziqiang Cao Wenjie Li 21 7 0 29 Nov 2022
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models Peter Henderson E. Mitchell Christopher D. Manning Dan Jurafsky Chelsea Finn 23 47 0 27 Nov 2022
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models Lei Wang Jian He Xingdong Xu Ning Liu Hui-juan Liu 33 2 0 27 Nov 2022
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations Amanpreet Singh Mike DÁrcy Arman Cohan Doug Downey Sergey Feldman 14 79 0 23 Nov 2022
Semi-Supervised Lifelong Language Learning Ying Zhao Yinhe Zheng Yu Bowen Zhiliang Tian Dongkyu Lee Jian Sun Haiyang Yu Yongbin Li N. Zhang CLL KELM 32 3 0 23 Nov 2022
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval Siteng Huang Biao Gong Yulin Pan Jianwen Jiang Yiliang Lv Yuyuan Li Donglin Wang VLM VPVLM 19 41 0 23 Nov 2022
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 19 11 0 23 Nov 2022
Continual Learning of Natural Language Processing Tasks: A Survey Zixuan Ke Bin Liu KELM CLL VLM 21 68 0 23 Nov 2022
Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models Mark Rofin Nikita Balagansky Daniil Gavrilov MoMe KELM 36 5 0 22 Nov 2022
Legal and Political Stance Detection of SCOTUS Language Noah Bergam Emily Allaway Kathleen McKeown AILaw ELM 22 5 0 21 Nov 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 42 1 0 21 Nov 2022
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition Guimin Hu Ting-En Lin Yi Zhao Guangming Lu Yuchuan Wu Yongbin Li 28 110 0 21 Nov 2022
CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering Yao Zhang Haokun Chen A. Frikha Yezi Yang Denis Krompass Gengyuan Zhang Jindong Gu Volker Tresp VLM LRM 16 7 0 19 Nov 2022
Scaling Native Language Identification with Transformer Adapters Ahmet Uluslu Gerold Schneider 27 6 0 18 Nov 2022
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 45 36 0 17 Nov 2022
Prompt Tuning for Parameter-efficient Medical Image Segmentation Marc Fischer Alexander Bartler Bin Yang SSeg 16 18 0 16 Nov 2022
Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations Linlin Liu Xingxuan Li Megh Thakkar Xin Li Shafiq R. Joty Luo Si Lidong Bing 27 2 0 16 Nov 2022
On the Compositional Generalization Gap of In-Context Learning Arian Hosseini Ankit Vani Dzmitry Bahdanau Alessandro Sordoni Aaron C. Courville 19 24 0 15 Nov 2022
FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers Jinyu Chen Wenchao Xu Song Guo Junxiao Wang Jie M. Zhang Haozhao Wang FedML 28 32 0 15 Nov 2022
An Overview on Controllable Text Generation via Variational Auto-Encoders Haoqin Tu Yitong Li BDL 19 2 0 15 Nov 2022
Finding Skill Neurons in Pre-trained Transformer-based Language Models Xiaozhi Wang Kaiyue Wen Zhengyan Zhang Lei Hou Zhiyuan Liu Juanzi Li MILM MoE 21 50 0 14 Nov 2022
BERT-Deep CNN: State-of-the-Art for Sentiment Analysis of COVID-19 Tweets Javad Hassannataj Joloudari Sadiq Hussain M. Nematollahi Rouhollah Bagheri Fatemeh Fazl R. Alizadehsani Reza Lashgari Ashis Talukder 18 38 0 04 Nov 2022
Two-stage LLM Fine-tuning with Less Specialization and More Generalization Yihan Wang Si Si Daliang Li Michal Lukasik Felix X. Yu Cho-Jui Hsieh Inderjit S Dhillon Sanjiv Kumar 38 29 0 01 Nov 2022
Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakers Cheng-Ping Hsieh Subhankar Ghosh Boris Ginsburg 41 18 0 01 Nov 2022
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks Rochelle Choenni Dan Garrette Ekaterina Shutova 24 2 0 31 Oct 2022
Domain Curricula for Code-Switched MT at MixMT 2022 Lekan Raheem Maab Elrashid 19 1 0 31 Oct 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 19 117 0 31 Oct 2022
GPS: Genetic Prompt Search for Efficient Few-shot Learning Hanwei Xu Yujun Chen Yulun Du Nan Shao Yanggang Wang Haiyu Li Zhilin Yang VLM 14 28 0 31 Oct 2022
Differentiable Data Augmentation for Contrastive Sentence Representation Learning Tianduo Wang Wei Lu SSL 32 9 0 29 Oct 2022
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition Steven Vander Eeckt Hugo Van hamme CLL MoMe 58 14 0 27 Oct 2022
Learning Better Intent Representations for Financial Open Intent Classification Xianzhi Li Will Aitken Xiao-Dan Zhu Stephen W. Thomas AIFin 4 8 0 25 Oct 2022
Exploring Mode Connectivity for Pre-trained Language Models Yujia Qin Cheng Qian Jing Yi Weize Chen Yankai Lin Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 29 20 0 25 Oct 2022
Evaluating Parameter Efficient Learning for Generation Peng-Tao Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Wei Ping Nayeon Lee M. Shoeybi Bryan Catanzaro MoE 27 3 0 25 Oct 2022
Adapters for Enhanced Modeling of Multilingual Knowledge and Text Yifan Hou Wenxiang Jiao Mei-Jun Liu Carl Allen Zhaopeng Tu Mrinmaya Sachan 26 11 0 24 Oct 2022
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning Jing Yi Weize Chen Yujia Qin Yankai Lin Ning Ding Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 12 2 0 24 Oct 2022
Specializing Multi-domain NMT via Penalizing Low Mutual Information Jiyoung Lee H. Kim Hyun Chang Cho E. Choi Cheonbok Park 16 4 0 24 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 34 27 0 20 Oct 2022
Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers Wanjun Zhong Tingting Ma Jiahai Wang Jian Yin T. Zhao Chin-Yew Lin Nan Duan LRM CoGe 25 2 0 20 Oct 2022
Towards Sustainable Self-supervised Learning Shanghua Gao Pan Zhou Mingg-Ming Cheng Shuicheng Yan CLL 40 7 0 20 Oct 2022
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking Tim Baumgärtner Leonardo F. R. Ribeiro Nils Reimers Iryna Gurevych 27 6 0 19 Oct 2022
Revision Transformers: Instructing Language Models to Change their Values Felix Friedrich Wolfgang Stammer P. Schramowski Kristian Kersting KELM 30 6 0 19 Oct 2022
Continued Pretraining for Better Zero- and Few-Shot Promptability Zhaofeng Wu IV RobertL.Logan Pete Walsh Akshita Bhagia Dirk Groeneveld Sameer Singh Iz Beltagy VLM 31 12 0 19 Oct 2022