Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

33 / 383 papers shown

Title
Studying Strategically: Learning to Mask for Closed-book QA Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa OffRL 19 11 0 31 Dec 2020
Automated Lay Language Summarization of Biomedical Scientific Reviews Yue Guo Weijian Qiu Yizhong Wang T. Cohen 22 77 0 23 Dec 2020
MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification Te-Lin Wu Shikhar Singh S. Paul Gully A. Burns Nanyun Peng 22 18 0 16 Dec 2020
Causal BERT : Language models for causality detection between events expressed in text Vivek Khetan Roshni Ramnani M. Anand Shubhashis Sengupta Andrew E.Fano 14 43 0 10 Dec 2020
CrossNER: Evaluating Cross-Domain Named Entity Recognition Zihan Liu Yan Xu Tiezheng Yu Wenliang Dai Ziwei Ji Samuel Cahyawijaya Andrea Madotto Pascale Fung 68 142 0 08 Dec 2020
EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answering Momchil Hardalov Todor Mihaylov Dimitrina Zlatkova Yoan Dinkov Ivan Koychev Preslav Nakov AI4Ed ELM 31 50 0 05 Nov 2020
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias Marion Bartl Malvina Nissim Albert Gatt 14 122 0 27 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 93 142 0 24 Oct 2020
Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality Gustavo Aguilar Bryan McCann Tong Niu Nazneen Rajani N. Keskar Thamar Solorio 42 12 0 24 Oct 2020
HateBERT: Retraining BERT for Abusive Language Detection in English Tommaso Caselli Valerio Basile Jelena Mitrović Michael Granitzer 17 358 0 23 Oct 2020
An Analysis of Simple Data Augmentation for Named Entity Recognition Xiang Dai Heike Adel 30 194 0 22 Oct 2020
Technical Question Answering across Tasks and Domains W. Yu Lingfei Wu Yu Deng Qingkai Zeng R. Mahindru S. Guven Meng-Long Jiang 28 8 0 19 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 219 608 0 13 Oct 2020
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers Tom Hope Aida Amini David Wadden Madeleine van Zuylen Sravanthi Parasa Eric Horvitz Daniel S. Weld Roy Schwartz Hannaneh Hajishirzi 24 29 0 08 Oct 2020
Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models Thuy-Trang Vu Dinh Q. Phung Gholamreza Haffari 6 24 0 05 Oct 2020
Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media Xiang Dai Sarvnaz Karimi Ben Hachey Cécile Paris 11 35 0 02 Oct 2020
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds Prithviraj Ammanabrolu Jack Urbanek Margaret Li Arthur Szlam Tim Rocktaschel Jason Weston LM&Ro 13 44 0 01 Oct 2020
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank Ethan C. Chau Lucy H. Lin Noah A. Smith 19 15 0 29 Sep 2020
DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue Shikib Mehri Mihail Eric Dilek Z. Hakkani-Tür ELM 8 136 0 28 Sep 2020
A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support Ashish Sharma Adam S. Miner David C. Atkins Tim Althoff AI4MH 25 268 0 17 Sep 2020
Transformer Based Multi-Source Domain Adaptation Dustin Wright Isabelle Augenstein 13 52 0 16 Sep 2020
Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues Ruijian Xu Chongyang Tao Daxin Jiang Xueliang Zhao Dongyan Zhao Rui Yan 24 70 0 14 Sep 2020
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge Kai Sun Dian Yu Jianshu Chen Dong Yu Claire Cardie 25 12 0 12 Sep 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 17 215 0 16 Jul 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Vu A. Nguyen VLM 9 900 0 20 May 2020
Evidence Inference 2.0: More Data, Better Models Jay DeYoung Eric P. Lehman Benjamin E. Nye Iain J. Marshall Byron C. Wallace 9 68 0 08 May 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 22 91 0 24 Apr 2020
Train No Evil: Selective Masking for Task-Guided Pre-Training Yuxian Gu Zhengyan Zhang Xiaozhi Wang Zhiyuan Liu Maosong Sun 24 59 0 21 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 24 243 0 05 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,450 0 18 Mar 2020
Explaining Relationships Between Scientific Documents Kelvin Luu Xinyi Wu Rik Koncel-Kedziorski Kyle Lo Isabel Cachola Noah A. Smith 28 48 0 02 Feb 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016