Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 729 papers shown

Title
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang X. Wang 28 84 0 29 Mar 2022
Fine-tuning Image Transformers using Learnable Memory Mark Sandler A. Zhmoginov Max Vladymyrov Andrew Jackson ViT 21 47 0 29 Mar 2022
Pathways: Asynchronous Distributed Dataflow for ML P. Barham Aakanksha Chowdhery J. Dean Sanjay Ghemawat Steven Hand ... Parker Schuh Ryan Sepassi Laurent El Shafey C. A. Thekkath Yonghui Wu GNN MoE 45 126 0 23 Mar 2022
Meta-attention for ViT-backed Continual Learning Mengqi Xue Haofei Zhang Jie Song Mingli Song CLL 20 41 0 22 Mar 2022
Continual Sequence Generation with Adaptive Compositional Modules Yanzhe Zhang Xuezhi Wang Diyi Yang KELM CLL 37 40 0 20 Mar 2022
$Meta-X$_{NLG}$: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation$ Meta-X $_{NLG}$ : A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation Kaushal Kumar Maurya M. Desarkar 14 8 0 19 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 24 20 0 15 Mar 2022
Graph Pre-training for AMR Parsing and Generation Xuefeng Bai Yulong Chen Yue Zhang SSL 41 98 0 15 Mar 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 15 196 0 14 Mar 2022
Continual Prompt Tuning for Dialog State Tracking Qi Zhu Bing Li Fei Mi Xiaoyan Zhu Minlie Huang CLL KELM 32 57 0 13 Mar 2022
Memory Efficient Continual Learning with Transformers B. Ermiş Giovanni Zappella Martin Wistuba Aditya Rawal Cédric Archambeau CLL 21 42 0 09 Mar 2022
Adaptor: Objective-Centric Adaptation Framework for Language Models Michal vStefánik Vít Novotný Nikola Groverová Petr Sojka 25 10 0 08 Mar 2022
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks Zhengkun Zhang Wenya Guo Xiaojun Meng Yasheng Wang Yadao Wang Xin Jiang Qun Liu Zhenglu Yang 26 15 0 08 Mar 2022
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models Shengnan An Yifei Li Zeqi Lin Qian Liu Bei Chen Qiang Fu Weizhu Chen Nanning Zheng Jian-Guang Lou VLM AAML 34 39 0 07 Mar 2022
$$\mathcal{Y}$-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning$ $\mathcal{Y}$ -Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning Yitao Liu Chen An Xipeng Qiu 27 17 0 20 Feb 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 88 0 16 Feb 2022
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation Tao Ge Si-Qing Chen Furu Wei MoE 22 21 0 16 Feb 2022
Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding Peter Sullivan Toshiko Shibano Muhammad Abdul-Mageed 34 11 0 10 Feb 2022
Can Open Domain Question Answering Systems Answer Visual Knowledge Questions? Jiawen Zhang Abhijit Mishra Avinesh P.V.S Siddharth Patwardhan Sachin Agarwal 24 0 0 09 Feb 2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Boxin Wang Wei Ping Chaowei Xiao P. Xu M. Patwary M. Shoeybi Bo-wen Li Anima Anandkumar Bryan Catanzaro 9 64 0 08 Feb 2022
Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition Bethan Thomas Samuel Kessler S. Karout 16 70 0 07 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 45 850 0 07 Feb 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 31 101 0 15 Jan 2022
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning Utku Evci Vincent Dumoulin Hugo Larochelle Michael C. Mozer 23 83 0 10 Jan 2022
Black-Box Tuning for Language-Model-as-a-Service Tianxiang Sun Yunfan Shao Hong Qian Xuanjing Huang Xipeng Qiu VLM 48 255 0 10 Jan 2022
TransLog: A Unified Transformer-based Framework for Log Anomaly Detection Hongcheng Guo Xin-Xue Lin Jian Yang Yi Zhuang Jiaqi Bai Tieqiao Zheng Bo Zhang Zhoujun Li 22 19 0 31 Dec 2021
Efficient Large Scale Language Modeling with Mixtures of Experts Mikel Artetxe Shruti Bhosale Naman Goyal Todor Mihaylov Myle Ott ... Jeff Wang Luke Zettlemoyer Mona T. Diab Zornitsa Kozareva Ves Stoyanov MoE 52 188 0 20 Dec 2021
Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages Hariom A. Pandya Bhavik Ardeshna Brijesh S. Bhatt 16 6 0 18 Dec 2021
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 23 42 0 16 Dec 2021
Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation Yu Li Baolin Peng Yelong Shen Yi Mao Lars Liden Zhou Yu Jianfeng Gao 14 53 0 15 Dec 2021
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Mohit Bansal VLM VPVLM 27 342 0 13 Dec 2021
Discourse-Aware Soft Prompting for Text Generation Marjan Ghazvininejad Vladimir Karpukhin Vera Gor Asli Celikyilmaz 23 6 0 10 Dec 2021
Pruning Pretrained Encoders with a Multitask Objective Patrick Xia Richard Shin 42 0 0 10 Dec 2021
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks Zixuan Ke Hu Xu Bing-Quan Liu CLL 235 84 0 06 Dec 2021
CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks Zixuan Ke Bing-Quan Liu Hu Xu Lei Shu CLL 25 55 0 05 Dec 2021
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning Zixuan Ke Bing-Quan Liu Nianzu Ma Hu Xu Lei Shu CLL 181 123 0 05 Dec 2021
VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts Longtian Qiu Renrui Zhang Ziyu Guo Wei Zhang Zilu Guo Ziyao Zeng Guangnan Zhang VLM CLIP 26 45 0 04 Dec 2021
Improving Differentially Private SGD via Randomly Sparsified Gradients Junyi Zhu Matthew B. Blaschko 21 5 0 01 Dec 2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio Valerii Likhosherstov Anurag Arnab K. Choromanski Mario Lucic Yi Tay Adrian Weller Mostafa Dehghani ViT 33 73 0 25 Nov 2021
Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture Daria Bakshandaeva Denis Dimitrov V.Ya. Arkhipkin Alex Shonenkov M. Potanin ... Mikhail Martynov Anton Voronov Vera Davydova E. Tutubalina Aleksandr Petiushko 33 0 0 22 Nov 2021
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples Linlin Liu Xin Li Ruidan He Lidong Bing Shafiq R. Joty Luo Si KELM 35 18 0 22 Nov 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 18 196 0 18 Nov 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 18 98 0 12 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 186 385 0 06 Nov 2021
Leveraging Sentiment Analysis Knowledge to Solve Emotion Detection Tasks Maude Nguyen-The Guillaume-Alexandre Bilodeau Jan Rockemann 8 4 0 05 Nov 2021
OpenPrompt: An Open-source Framework for Prompt-learning Ning Ding Shengding Hu Weilin Zhao Yulin Chen Zhiyuan Liu Haitao Zheng Maosong Sun VLM LLMAG 23 284 0 03 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 71 1,029 0 01 Nov 2021
Semi-Siamese Bi-encoder Neural Ranking Model Using Lightweight Fine-Tuning Euna Jung Jaekeol Choi Wonjong Rhee 17 13 0 28 Oct 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 32 98 0 25 Oct 2021
The Power of Prompt Tuning for Low-Resource Semantic Parsing Nathan Schucher Siva Reddy H. D. Vries VLM 101 36 0 16 Oct 2021