Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 771 papers shown

Title
Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures Zirui Fu Aleksandre Avaliani M. Donato 39 1 0 25 Mar 2023
SPEC: Summary Preference Decomposition for Low-Resource Abstractive Summarization Yi-Syuan Chen Yun-Zhu Song Hong-Han Shuai 33 6 0 24 Mar 2023
SwissBERT: The Multilingual Language Model for Switzerland Jannis Vamvas Johannes Graen Rico Sennrich 35 6 0 23 Mar 2023
Parameter-Efficient Sparse Retrievers and Rerankers using Adapters Vaishali Pal Carlos Lassance Hervé Déjean S. Clinchant 132 3 0 23 Mar 2023
Beyond Universal Transformer: block reusing with adaptor in Transformer for automatic speech recognition Haoyu Tang Zhaoyi Liu Chang Zeng Xinfeng Li 34 1 0 23 Mar 2023
A Region-Prompted Adapter Tuning for Visual Abductive Reasoning Hao Zhang Yeo Keat Ee Basura Fernando VLM 27 3 0 18 Mar 2023
Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey Huali Xu Shuaifeng Zhi Shuzhou Sun Vishal M. Patel Li Liu 32 13 0 15 Mar 2023
Diversity-Aware Meta Visual Prompting Qidong Huang Xiaoyi Dong Dongdong Chen Weiming Zhang Feifei Wang Gang Hua Neng H. Yu VLM VPVLM 41 52 0 14 Mar 2023
ViM: Vision Middleware for Unified Downstream Transferring Yutong Feng Biao Gong Jianwen Jiang Yiliang Lv Yujun Shen Deli Zhao Jingren Zhou 32 1 0 13 Mar 2023
An Overview on Language Models: Recent Developments and Outlook Chengwei Wei Yun Cheng Wang Bin Wang C.-C. Jay Kuo 25 42 0 10 Mar 2023
IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models Edoardo Mosca Daryna Dementieva Tohid Ebrahim Ajdari Maximilian Kummeth Kirill Gringauz Yutong Zhou Georg Groh 24 8 0 06 Mar 2023
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners Renrui Zhang Xiangfei Hu Bohao Li Siyuan Huang Hanqiu Deng Hongsheng Li Yu Qiao Peng Gao VLM MLLM 38 170 0 03 Mar 2023
Enhancing General Face Forgery Detection via Vision Transformer with Low-Rank Adaptation Chen Kong Haoliang Li Shiqi Wang ViT CVBM 39 15 0 02 Mar 2023
SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases Yanchen Liu Jing Yang Yan Chen Jing Liu Huaqin Wu MoE 44 2 0 28 Feb 2023
Language-Driven Representation Learning for Robotics Siddharth Karamcheti Suraj Nair Annie S. Chen Thomas Kollar Chelsea Finn Dorsa Sadigh Percy Liang LM&Ro SSL 38 145 0 24 Feb 2023
Active Prompting with Chain-of-Thought for Large Language Models Shizhe Diao Pengcheng Wang Yong Lin Tong Zhang ReLM KELM LLMAG LRM 29 119 0 23 Feb 2023
How Does In-Context Learning Help Prompt Tuning? Simeng Sun Yang Liu Dan Iter Chenguang Zhu Mohit Iyyer VLM 30 17 0 22 Feb 2023
Few-shot Multimodal Multitask Multilingual Learning Aman Chadha Vinija Jain 45 0 0 19 Feb 2023
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems Jiajun Deng Xurong Xie Tianzi Wang Mingyu Cui Boyang Xue Zengrui Jin Guinan Li Shujie Hu Xunying Liu 26 5 0 15 Feb 2023
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Alexander M. Fraser Jesse Dodge MoMe 21 65 0 14 Feb 2023
Towards Agile Text Classifiers for Everyone Maximilian Mozes Jessica Hoffmann Katrin Tomanek Muhamed Kouate Nithum Thain Ann Yuan Tolga Bolukbasi Lucas Dixon 34 13 0 13 Feb 2023
Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters Yawen Cui Zitong Yu Rizhao Cai Xuna Wang Alex C. Kot Li Liu CLL 28 7 0 12 Feb 2023
Flexible-modal Deception Detection with Audio-Visual Adapter Zhaoxu Li Zitong Yu Nithish Muthuchamy Selvaraj Xiaobao Guo Bingquan Shen A. Kong Alex C. Kot 29 2 0 11 Feb 2023
Distillation of encoder-decoder transformers for sequence labelling M. Farina D. Pappadopulo Anant Gupta Leslie Huang Ozan Irsoy Thamar Solorio VLM 103 3 0 10 Feb 2023
AIM: Adapting Image Models for Efficient Video Action Recognition Taojiannan Yang Yi Zhu Yusheng Xie Aston Zhang C. L. P. Chen Mu Li ViT 46 144 0 06 Feb 2023
Multipath agents for modular multitask ML systems Andrea Gesmundo 28 1 0 06 Feb 2023
CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets Zachary Novack Julian McAuley Zachary Chase Lipton Saurabh Garg VLM 24 79 0 06 Feb 2023
Interpretations of Domain Adaptations via Layer Variational Analysis H. Tseng Hsin-Yi Lin Kuo-Hsuan Hung Yu Tsao FAtt 33 3 0 03 Feb 2023
Efficient Domain Adaptation for Speech Foundation Models Bo-wen Li DongSeon Hwang Zhouyuan Huo Junwen Bai Guru Prakash ... K. Sim Yu Zhang Wei Han Trevor Strohman F. Beaufays AI4CE 36 23 0 03 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 28 47 0 02 Feb 2023
idT5: Indonesian Version of Multilingual T5 Transformer Mukhlish Fuadi A. Wibawa S. Sumpeno 11 6 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 11 2 0 01 Feb 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 18 45 0 28 Jan 2023
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling Kolby Nottingham Prithviraj Ammanabrolu Alane Suhr Yejin Choi Hannaneh Hajishirzi Sameer Singh Roy Fox LLMAG LM&Ro 42 77 0 28 Jan 2023
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning Hyunsoo Cho Choonghyun Park Junyeop Kim Hyuhng Joon Kim Kang Min Yoo Sang-goo Lee OODD 27 3 0 27 Jan 2023
One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER Xiang Chen Lei Li Q. Fei Ningyu Zhang Chuanqi Tan Yong-jia Jiang Fei Huang Huajun Chen 26 23 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
The Semantic Scholar Open Data Platform Rodney Michael Kinney Chloe Anastasiades Russell Authur Iz Beltagy Jonathan Bragg ... Caroline M Wu Jiangjiang Yang Angele Zamarron Madeleine van Zuylen Daniel S. Weld 16 93 0 24 Jan 2023
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning Malte Ostendorff Georg Rehm CLIP VLM CLL 41 23 0 23 Jan 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 22 18 0 22 Jan 2023
BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation Xiangyu Qin Zhiyu Wu J. Cui Ting Zhang Yanran Li Jian Luan Bin Wang L. xilinx Wang 32 26 0 17 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 15 100 0 16 Jan 2023
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing Chen Cecilia Liu Jonas Pfeiffer Ivan Vulić Iryna Gurevych CLL 24 9 0 13 Jan 2023
Multimodal Inverse Cloze Task for Knowledge-based Visual Question Answering Paul Lerner O. Ferret C. Guinaudeau 16 9 0 11 Jan 2023
Exploring Efficient Few-shot Adaptation for Vision Transformers C. Xu Siqian Yang Yabiao Wang Zhanxiong Wang Yanwei Fu Xiangyang Xue 32 16 0 06 Jan 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 31 59 0 04 Jan 2023
Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greek Georgios Paraskevopoulos Theodoros Kouzelis Georgios Rouvalis Athanasios Katsamanis V. Katsouros Alexandros Potamianos VLM 23 7 0 31 Dec 2022
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 28 18 0 23 Dec 2022
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning Zhiyang Xu Ying Shen Lifu Huang MLLM 32 110 0 21 Dec 2022
When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods Zhuo Zhang Yuanhang Yang Yong Dai Lizhen Qu Zenglin Xu FedML 33 65 0 20 Dec 2022