What to Pre-Train on? Efficient Intermediate Task Selection

16 April 2021

Papers citing "What to Pre-Train on? Efficient Intermediate Task Selection"

25 / 25 papers shown

Title
Model-Based Transfer Learning for Contextual Reinforcement Learning Jung-Hoon Cho Vindula Jayawardana Sirui Li Cathy Wu OffRL 45 0 0 08 Aug 2024
TaskWeb: Selecting Better Source Tasks for Multi-task NLP Joongwon Kim Akari Asai Gabriel Ilharco Hannaneh Hajishirzi 21 11 0 22 May 2023
ViM: Vision Middleware for Unified Downstream Transferring Yutong Feng Biao Gong Jianwen Jiang Yiliang Lv Yujun Shen Deli Zhao Jingren Zhou 32 1 0 13 Mar 2023
Detecting software vulnerabilities using Language Models Marwan Omar 24 11 0 23 Feb 2023
RobustNLP: A Technique to Defend NLP Models Against Backdoor Attacks Marwan Omar SILM AAML 23 0 0 18 Feb 2023
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preotiuc-Pietro Pengxiang Cheng FedML MoMe 15 211 0 19 Dec 2022
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors Hamish Ivison Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 31 19 0 01 Dec 2022
Exploring Mode Connectivity for Pre-trained Language Models Yujia Qin Cheng Qian Jing Yi Weize Chen Yankai Lin Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 27 20 0 25 Oct 2022
Visual Comparison of Language Model Adaptation R. Sevastjanova E. Cakmak Shauli Ravfogel Ryan Cotterell Mennatallah El-Assady VLM 41 16 0 17 Aug 2022
Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations Qian Yang Yunxin Li Baotian Hu Lin Ma Yuxin Ding Min Zhang 25 10 0 23 Jul 2022
When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning Orion Weller Kevin Seppi Matt Gardner 8 21 0 17 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 40 138 0 12 May 2022
Efficient Few-Shot Fine-Tuning for Opinion Summarization Arthur Bravzinskas Ramesh Nallapati Mohit Bansal Markus Dreyer 17 24 0 04 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 27 63 0 03 May 2022
Memory Efficient Continual Learning with Transformers B. Ermiş Giovanni Zappella Martin Wistuba Aditya Rawal Cédric Archambeau CLL 21 42 0 09 Mar 2022
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 18 98 0 12 Nov 2021
xGQA: Cross-Lingual Visual Question Answering Jonas Pfeiffer Gregor Geigle Aishwarya Kamath Jan-Martin O. Steitz Stefan Roth Ivan Vulić Iryna Gurevych 26 56 0 13 Sep 2021
Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training Momchil Hardalov Arnav Arora Preslav Nakov Isabelle Augenstein 22 58 0 13 Sep 2021
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting Xiang Chen Lei Li Shumin Deng Chuanqi Tan Changliang Xu Fei Huang Luo Si Huajun Chen Ningyu Zhang VLM 34 65 0 31 Aug 2021
A linearized framework and a new benchmark for model selection for fine-tuning Aditya Deshpande Alessandro Achille Avinash Ravichandran Hao Li L. Zancato Charless C. Fowlkes Rahul Bhotika Stefano Soatto Pietro Perona ALM 107 46 0 29 Jan 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavas 31 27 0 11 Dec 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 46 254 0 22 Oct 2020
Which Model to Transfer? Finding the Needle in the Growing Haystack Cédric Renggli André Susano Pinto Luka Rimanic J. Puigcerver C. Riquelme Ce Zhang Mario Lucic 23 23 0 13 Oct 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018