Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?

1 May 2020

Papers citing "Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?"

41 / 41 papers shown

Title
MSciNLI: A Diverse Benchmark for Scientific Natural Language Inference Mobashir Sadat Cornelia Caragea 32 4 0 11 Apr 2024
Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study Myrthe Reuver Suzan Verberne Antske Fokkens 32 1 0 05 Apr 2024
PartSTAD: 2D-to-3D Part Segmentation Task Adaptation Hyunjin Kim Minhyuk Sung 45 8 0 11 Jan 2024
TaskWeb: Selecting Better Source Tasks for Multi-task NLP Joongwon Kim Akari Asai Gabriel Ilharco Hannaneh Hajishirzi 21 11 0 22 May 2023
I2I: Initializing Adapters with Improvised Knowledge Tejas Srinivasan Furong Jia Mohammad Rostami Jesse Thomason CLL 24 6 0 04 Apr 2023
Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective Jongwoo Ko Seungjoon Park Minchan Jeong S. Hong Euijai Ahn Duhyeuk Chang Se-Young Yun 21 6 0 03 Feb 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
Towards Proactively Forecasting Sentence-Specific Information Popularity within Online News Documents Sayar Ghosh Roy Anshul Padhi Risubh Jain Manish Gupta Vasudeva Varma AI4TS 23 2 0 31 Dec 2022
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preotiuc-Pietro Pengxiang Cheng FedML MoMe 13 211 0 19 Dec 2022
Exploring Mode Connectivity for Pre-trained Language Models Yujia Qin Cheng Qian Jing Yi Weize Chen Yankai Lin Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 27 20 0 25 Oct 2022
Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models Syrielle Montariol Arij Riabi Djamé Seddah 16 10 0 24 Oct 2022
Enhancing Tabular Reasoning with Pattern Exploiting Training Abhilash Shankarampeta Vivek Gupta Shuo Zhang LMTD RALM ReLM 60 6 0 21 Oct 2022
An Exploration of Data Efficiency in Intra-Dataset Task Transfer for Dialog Understanding Josiah Ross Luke Yoffe Alon Albalak William Yang Wang 30 0 0 21 Oct 2022
SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data Ching-Yun Ko Pin-Yu Chen Jeet Mohapatra Payel Das Lucani E. Daniel 19 3 0 06 Oct 2022
Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Prediction Michiel van der Meer Myrthe Reuver Urja Khurana Lea Krause Selene Báez Santamaría 37 14 0 19 Sep 2022
Summarizing Patients Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models Yanjun Gao Dmitriy Dligach T. Miller Dongfang Xu M. Churpek Majid Afshar AI4MH 22 36 0 17 Aug 2022
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts Qinyuan Ye Juan Zha Xiang Ren MoE 15 12 0 25 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 27 63 0 03 May 2022
Developmental Negation Processing in Transformer Language Models Antonio Laverghetta John Licato LRM 25 5 0 29 Apr 2022
Fusing finetuned models for better pretraining Leshem Choshen Elad Venezian Noam Slonim Yoav Katz FedML AI4CE MoMe 41 86 0 06 Apr 2022
SciNLI: A Corpus for Natural Language Inference on Scientific Text Mobashir Sadat Cornelia Caragea AILaw 17 35 0 13 Mar 2022
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt Lianzhe Huang Shuming Ma Dongdong Zhang Furu Wei Houfeng Wang VLM LRM 14 32 0 23 Feb 2022
Probing Pretrained Models of Source Code Sergey Troshin Nadezhda Chirkova ELM 25 38 0 16 Feb 2022
Assemble Foundation Models for Automatic Code Summarization Jian Gu P. Salza H. Gall 28 34 0 13 Jan 2022
QuALITY: Question Answering with Long Input Texts, Yes! Richard Yuanzhe Pang Alicia Parrish Nitish Joshi Nikita Nangia Jason Phang ... Vishakh Padmakumar Johnny Ma Jana Thompson He He Sam Bowman RALM 25 141 0 16 Dec 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 18 98 0 12 Nov 2021
Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks Wang Zhu Peter Shaw Tal Linzen Fei Sha 27 7 0 09 Nov 2021
The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding Archiki Prasad Mohammad Ali Rehan Shreyasi Pathak P. Jyothi 16 9 0 21 Jul 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 24 63 0 27 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 30 2,744 0 15 Jun 2021
Question Answering Infused Pre-training of General-Purpose Contextualized Representations Robin Jia M. Lewis Luke Zettlemoyer 18 28 0 15 Jun 2021
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 209 179 0 18 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 18 164 0 16 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 27 27 0 13 Apr 2021
What Will it Take to Fix Benchmarking in Natural Language Understanding? Samuel R. Bowman George E. Dahl ELM ALM 28 156 0 05 Apr 2021
LogME: Practical Assessment of Pre-trained Models for Transfer Learning Kaichao You Yong Liu Jianmin Wang Mingsheng Long 16 178 0 22 Feb 2021
Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers Arthur Brack Anett Hoppe Pascal Buschermöhle Ralph Ewerth 19 18 0 11 Feb 2021
A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters Mengjie Zhao Yi Zhu Ehsan Shareghi Ivan Vulić Roi Reichart Anna Korhonen Hinrich Schütze 19 64 0 31 Dec 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 21 136 0 10 Nov 2020
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 882 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018