Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

50 / 352 papers shown

Title
UPB at SemEval-2022 Task 5: Enhancing UNITER with Image Sentiment and Graph Convolutional Networks for Multimedia Automatic Misogyny Identification Andrei Paraschiv M. Dascalu Dumitru-Clementin Cercel 19 3 0 29 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 34 9 0 27 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference Weijia Shi Julian Michael Suchin Gururangan Luke Zettlemoyer RALM VLM 13 32 0 27 May 2022
Quark: Controllable Text Generation with Reinforced Unlearning Ximing Lu Sean Welleck Jack Hessel Liwei Jiang Lianhui Qin Peter West Prithviraj Ammanabrolu Yejin Choi MU 47 206 0 26 May 2022
The Document Vectors Using Cosine Similarity Revisited Bingyu Zhang N. Arefyev 19 9 0 26 May 2022
Detecting Label Errors by using Pre-Trained Language Models Derek Chong Jenny Hong Christopher D. Manning NoLa 38 21 0 25 May 2022
Gradient-Based Constrained Sampling from Language Models Sachin Kumar Biswajit Paria Yulia Tsvetkov BDL 28 53 0 25 May 2022
When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning Orion Weller Kevin Seppi Matt Gardner 8 21 0 17 May 2022
Improving Contextual Representation with Gloss Regularized Pre-training Yu Lin Zhecheng An Peihao Wu Zejun Ma 19 5 0 13 May 2022
Clinical Prompt Learning with Frozen Language Models Niall Taylor Yi Zhang Dan W Joyce A. Nevado-Holgado Andrey Kormilitzin VLM LM&MA 16 31 0 11 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 25 1 0 09 May 2022
Improving negation detection with negation-focused pre-training Thinh Hung Truong Timothy Baldwin Trevor Cohn Karin Verspoor 25 20 0 09 May 2022
A Dataset for N-ary Relation Extraction of Drug Combinations Aryeh Tiktinsky Vijay Viswanathan Danna Niezni D. Azagury Y. Shamay Hillel Taub-Tabib Tom Hope Yoav Goldberg 29 18 0 04 May 2022
Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding Yanfeng Chang Yun-Nung Chen 15 9 0 02 May 2022
POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection Yujian Liu Xinliang Frederick Zhang David Wegsman Nick Beauchamp Lu Wang 30 71 0 02 May 2022
Crude Oil-related Events Extraction and Processing: A Transfer Learning Approach Meisin Lee Lay-Ki Soon Eu-Gene Siew 14 0 0 01 May 2022
Detoxifying Language Models with a Toxic Corpus Yoon A Park Frank Rudzicz 11 6 0 30 Apr 2022
SkillSpan: Hard and Soft Skill Extraction from English Job Postings Mike Zhang Kristian Nørgaard Jensen Sif Dam Sonniks Barbara Plank 12 53 0 27 Apr 2022
Modular Domain Adaptation Junshen K. Chen Dallas Card Dan Jurafsky 17 1 0 26 Apr 2022
Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extraction Hui-Syuan Yeh Thomas Lavergne Pierre Zweigenbaum 19 10 0 21 Apr 2022
A Corpus for Understanding and Generating Moral Stories Jian-Yu Guan Ziqi Liu Minlie Huang 24 9 0 20 Apr 2022
Synthetic Target Domain Supervision for Open Retrieval QA Revanth Reddy Gangi Reddy Bhavani Iyer Md Arafat Sultan Rong Zhang Avirup Sil Vittorio Castelli Radu Florian Salim Roukos OOD 28 12 0 20 Apr 2022
Zero-shot Entity and Tweet Characterization with Designed Conditional Prompts and Contexts S. Srivatsa Tushar Mohan Kumari Neha Nishchay Malakar Ponnurangam Kumaraguru S. Srinivasa 20 0 0 18 Apr 2022
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base Cunxiang Wang Fuli Luo Yanyang Li Runxin Xu Fei Huang Yue Zhang KELM 22 2 0 17 Apr 2022
Improving Passage Retrieval with Zero-Shot Question Generation Devendra Singh Sachan M. Lewis Mandar Joshi Armen Aghajanyan Wen-tau Yih J. Pineau Luke Zettlemoyer OOD LRM 19 155 0 15 Apr 2022
BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model Hongyi Yuan Zheng Yuan Ruyi Gan Jiaxing Zhang Yutao Xie Sheng Yu LM&MA 22 122 0 08 Apr 2022
Fusing finetuned models for better pretraining Leshem Choshen Elad Venezian Noam Slonim Yoav Katz FedML AI4CE MoMe 41 86 0 06 Apr 2022
PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations L. D. Prasad Sreyan Ghosh S. Umesh 17 12 0 31 Mar 2022
Neural Pipeline for Zero-Shot Data-to-Text Generation Zdeněk Kasner Ondrej Dusek 16 33 0 30 Mar 2022
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia Alham Fikri Aji Genta Indra Winata Fajri Koto Samuel Cahyawijaya Ade Romadhony ... David Moeljadi Radityo Eko Prasojo Timothy Baldwin Jey Han Lau Sebastian Ruder 38 98 0 24 Mar 2022
Mix and Match: Learning-free Controllable Text Generation using Energy Language Models Fatemehsadat Mireshghallah Kartik Goyal Taylor Berg-Kirkpatrick 30 78 0 24 Mar 2022
From Stance to Concern: Adaptation of Propositional Analysis to New Tasks and Domains Brodie Mather Bonnie J. Dorr Adam Dalton William de Beaumont Owen Rambow Sonja M. Schmer-Galunder 20 8 0 20 Mar 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Representation Learning for Resource-Constrained Keyphrase Generation Di Wu Wasi Uddin Ahmad Sunipa Dev Kai-Wei Chang 36 17 0 15 Mar 2022
ELLE: Efficient Lifelong Pre-training for Emerging Data Yujia Qin Jiajie Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou 19 67 0 12 Mar 2022
LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval Canwen Xu Daya Guo Nan Duan Julian McAuley RALM VLM 16 45 0 11 Mar 2022
Adaptor: Objective-Centric Adaptation Framework for Language Models Michal vStefánik Vít Novotný Nikola Groverová Petr Sojka 20 10 0 08 Mar 2022
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots Wenting Zhao Ye Liu Yao Wan Philip S. Yu 21 11 0 01 Mar 2022
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation Xiang Hu Haitao Mi Liang Li Gerard de Melo 21 13 0 01 Mar 2022
Reward Modeling for Mitigating Toxicity in Transformer-based Language Models Farshid Faal K. Schmitt Jia Yuan Yu 11 25 0 19 Feb 2022
Automated Attack Synthesis by Extracting Finite State Machines from Protocol Specification Documents Maria Leonor Pacheco Max von Hippel Ben Weintraub Dan Goldwasser Cristina Nita-Rotaru 14 30 0 18 Feb 2022
Semantic-Oriented Unlabeled Priming for Large-Scale Language Models Yanchen Liu Timo Schick Hinrich Schütze VLM 25 15 0 12 Feb 2022
AdaPrompt: Adaptive Model Training for Prompt-based NLP Yulong Chen Yang Liu Li Dong Shuohang Wang Chenguang Zhu Michael Zeng Yue Zhang VLM 8 45 0 10 Feb 2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Boxin Wang Wei Ping Chaowei Xiao P. Xu M. Patwary M. Shoeybi Bo-wen Li Anima Anandkumar Bryan Catanzaro 4 64 0 08 Feb 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... M. Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 55 728 0 28 Jan 2022
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis Shamsuddeen Hassan Muhammad David Ifeoluwa Adelani Sebastian Ruder I. Ahmad Idris Abdulmumin ... Chris C. Emezue Saheed Abdul Anuoluwapo Aremu Alipio Jeorge P. Brazdil 35 95 0 20 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 17 101 0 15 Jan 2022
Assemble Foundation Models for Automatic Code Summarization Jian Gu P. Salza H. Gall 25 34 0 13 Jan 2022
Neural Architectures for Biological Inter-Sentence Relation Extraction Enrique Noriega-Atala Peter Lovett Clayton T. Morrison Mihai Surdeanu NAI 25 3 0 17 Dec 2021
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 23 42 0 16 Dec 2021