Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 771 papers shown

Title
Named Entity Recognition Under Domain Shift via Metric Learning for Life Sciences Hongyi Liu Qingyun Wang Payam Karisani Heng Ji 16 1 0 19 Jan 2024
Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR Junwen Bai Bo-wen Li Qiujia Li Tara N. Sainath Trevor Strohman 30 3 0 17 Jan 2024
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models Yae Jee Cho Luyang Liu Zheng Xu Aldi Fahrezi Gauri Joshi 27 45 0 12 Jan 2024
Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion Ruilin Luo Tianle Gu Haoling Li Junzhe Li Zicheng Lin Jiayi Li Yujiu Yang AI4CE 28 7 0 11 Jan 2024
Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapters Kenichi Fujita Hiroshi Sato Takanori Ashihara Hiroki Kanagawa Marc Delcroix Takafumi Moriya Yusuke Ijima 31 8 0 10 Jan 2024
LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training Khoi M. Le Trinh Pham Tho Quan A. Luu 19 7 0 09 Jan 2024
LLM Augmented LLMs: Expanding Capabilities through Composition Rachit Bansal Bidisha Samanta Siddharth Dalmia Nitish Gupta Shikhar Vashishth Sriram Ganapathy Abhishek Bapna Prateek Jain Partha P. Talukdar CLL 21 34 0 04 Jan 2024
Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation Zixian Guo Yuxiang Wei Ming-Yu Liu Zhilong Ji Jinfeng Bai Yiwen Guo Wangmeng Zuo VLM 31 8 0 26 Dec 2023
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 24 5 0 25 Dec 2023
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel S. Pokutta VLM 50 10 0 23 Dec 2023
Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion Katrin Tomanek Shanqing Cai Subhashini Venugopalan 32 1 0 21 Dec 2023
Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation Jiayu Lin Rong Ye Meng Han Qi Zhang Ruofei Lai Xinyu Zhang Zhao Cao Xuanjing Huang Zhongyu Wei LRM 28 6 0 21 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 19 7 0 19 Dec 2023
MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA Lang Yu Qin Chen Jie Zhou Liang He KELM 15 45 0 19 Dec 2023
SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing Zeyinzi Jiang Chaojie Mao Yulin Pan Zhen Han Jingfeng Zhang 24 28 0 18 Dec 2023
LaViP:Language-Grounded Visual Prompts Nilakshan Kunananthaseelan Jing Zhang Mehrtash Harandi VLM 17 0 0 18 Dec 2023
Attention-Guided Adaptation for Code-Switching Speech Recognition Bobbi Aditya Mahdin Rohmatillah Liang-Hsuan Tai Jen-Tzung Chien 26 8 0 14 Dec 2023
Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models Kyra Ahrens Hans Hergen Lehmann Jae Hee Lee Stefan Wermter CLL 35 7 0 13 Dec 2023
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models Junhao Zheng Shengjie Qiu Qianli Ma 25 9 0 13 Dec 2023
Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach Xingshuai Huang Di Wu Benoit Boulet OffRL 19 2 0 12 Dec 2023
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 32 0 0 12 Dec 2023
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 26 3 0 29 Nov 2023
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames Shuming Liu Chen-Da Liu-Zhang Chen Zhao Bernard Ghanem 33 25 0 28 Nov 2023
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA Damjan Kalajdzievski ALM 22 77 0 28 Nov 2023
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer Junyuan Hong Jiachen T. Wang Chenhui Zhang Zhangheng Li Bo-wen Li Zhangyang Wang 45 29 0 27 Nov 2023
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation Yonatan Sverdlov Shimon Ullman 23 0 0 26 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 45 11 0 23 Nov 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 36 43 0 20 Nov 2023
PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization Joseph Peper Wenzhao Qiu Lu Wang 23 0 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 21 41 0 16 Nov 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 24 14 0 15 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 21 27 0 15 Nov 2023
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks Hao Peng Xiaozhi Wang Jianhui Chen Weikai Li Y. Qi ... Zhili Wu Kaisheng Zeng Bin Xu Lei Hou Juanzi Li 31 28 0 15 Nov 2023
On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model Nohil Park Joonsuk Park Kang Min Yoo Sungroh Yoon 26 3 0 14 Nov 2023
Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer Dongping Chen 33 3 0 12 Nov 2023
DONUT-hole: DONUT Sparsification by Harnessing Knowledge and Optimizing Learning Efficiency Azhar Shaikh Michael Cochez Denis Diachkov Michiel de Rijcke Sahar Yousefi 25 0 0 09 Nov 2023
AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification Yongxin Huang Kexin Wang Sourav Dutta Raj Nath Patel Goran Glavas Iryna Gurevych VLM 18 4 0 01 Nov 2023
Unlearn What You Want to Forget: Efficient Unlearning for LLMs Jiaao Chen Diyi Yang MU 27 136 0 31 Oct 2023
Unified Representation for Non-compositional and Compositional Expressions Ziheng Zeng Suma Bhat 17 3 0 29 Oct 2023
Punica: Multi-Tenant LoRA Serving Lequn Chen Zihao Ye Yongji Wu Danyang Zhuo Luis Ceze Arvind Krishnamurthy 44 34 0 28 Oct 2023
Parameter-Efficient Methods for Metastases Detection from Clinical Notes Maede Ashofteh Barabadi Xiaodan Zhu Wai-Yip Chan Amber L. Simpson Richard K G Do 21 1 0 27 Oct 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 24 5 0 26 Oct 2023
Cascaded Multi-task Adaptive Learning Based on Neural Architecture Search Yingying Gao Shilei Zhang Zihao Cui Chao Deng Junlan Feng 15 0 0 23 Oct 2023
Scalable Neural Network Kernels Arijit Sehanobish Krzysztof Choromanski Yunfan Zhao Kumar Avinava Dubey Valerii Likhosherstov 36 4 0 20 Oct 2023
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model Abhijith Chintam Rahel Beloch Willem H. Zuidema Michael Hanna Oskar van der Wal 28 16 0 19 Oct 2023
Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition Hillary Ngai Rohan Agrawal Neeraj Gaur Ronny Huang Parisa Haghani P. M. Mengibar MoMe 34 0 0 17 Oct 2023
Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 16 3 0 17 Oct 2023
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning Chong Li Shaonan Wang Yunhao Zhang Jiajun Zhang Chengqing Zong 30 4 0 16 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 35 117 0 16 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 17 6 0 16 Oct 2023