BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based
Masked Language-models

v1v2v3v4v5 (latest)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

Annual Meeting of the Association for Computational Linguistics (ACL), 2021

18 June 2021

Shauli Ravfogel

ArXiv (abs)PDF HTML

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

10 / 960 papers shown

Title
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 526 437 0 13 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer LearningInternational Conference on Learning Representations (ICLR), 2021 Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 495 1,083 0 08 Oct 2021
MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer Ilias Chalkidis Manos Fergadiotis Ion Androutsopoulos AILaw 335 129 0 02 Sep 2021
DEMix Layers: Disentangling Domains for Modular Language ModelingNorth American Chapter of the Association for Computational Linguistics (NAACL), 2021 Suchin Gururangan Michael Lewis Ari Holtzman Noah A. Smith Luke Zettlemoyer KELM MoE 306 159 0 11 Aug 2021
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models Robert L Logan IV Ivana Balavzević Eric Wallace Fabio Petroni Sameer Singh Sebastian Riedel VPVLM 227 230 0 24 Jun 2021
LoRA: Low-Rank Adaptation of Large Language ModelsInternational Conference on Learning Representations (ICLR), 2021 J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 1.5K 14,854 0 17 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter LayersNeural Information Processing Systems (NeurIPS), 2021 Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 360 576 0 08 Jun 2021
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence EncodersConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Fangyu Liu Ivan Vulić Anna Korhonen Nigel Collier VLM OffRL 277 129 0 16 Apr 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language ModelsNeural Information Processing Systems (NeurIPS), 2021 Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 377 247 0 03 Feb 2021
AdapterFusion: Non-Destructive Task Composition for Transfer LearningConference of the European Chapter of the Association for Computational Linguistics (EACL), 2020 Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Dong Wang Iryna Gurevych CLL MoMe 732 1,009 0 01 May 2020