Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

50 / 382 papers shown

Title
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts Ashutosh Baheti Maarten Sap Alan Ritter Mark O. Riedl 16 84 0 26 Aug 2021
Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition Ying Zhang Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 14 10 0 22 Jul 2021
Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification Junghoon Lee Jounghee Kim Pilsung Kang VLM 9 5 0 22 Jul 2021
Small-Text: Active Learning for Text Classification in Python Christopher Schröder Lydia Muller A. Niekler Martin Potthast CLIP VLM AI4CE 31 23 0 21 Jul 2021
The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding Archiki Prasad Mohammad Ali Rehan Shreyasi Pathak P. Jyothi 11 9 0 21 Jul 2021
Adaptive Transfer Learning on Graph Neural Networks Xueting Han Zhenhuan Huang Bang An Jing Bai 22 59 0 19 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 26 17 0 04 Jul 2021
Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation Anne Lauscher Henning Wachsmuth Iryna Gurevych Goran Glavavs 25 31 0 01 Jul 2021
Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature Yu-Chiang Frank Wang Jinchao Li Tristan Naumann Chenyan Xiong Hao Cheng ... Yang Qin Eric Horvitz Paul N. Bennett Jianfeng Gao Hoifung Poon OOD 25 13 0 25 Jun 2021
GAIA: A Transfer Learning System of Object Detection that Fits Your Needs Xingyuan Bu Junran Peng Junjie Yan T. Tan Zhaoxiang Zhang ObjD VLM 20 53 0 21 Jun 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 19 222 0 18 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 25 27 0 16 Jun 2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data Haoming Jiang Danqing Zhang Tianyu Cao Bing Yin T. Zhao NoLa 19 44 0 16 Jun 2021
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 29 179 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 35 813 0 14 Jun 2021
CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing Sai Muralidhar Jayanthi Kavya Nerella Khyathi Raghavi Chandu A. Black MoE 23 8 0 10 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 29 11 0 10 Jun 2021
Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition Yihong Dong Ying Peng Muqiao Yang Songtao Lu Qingjiang Shi 38 9 0 05 Jun 2021
MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding Jia-Chen Gu Chongyang Tao Zhenhua Ling Can Xu Xiubo Geng Daxin Jiang 11 53 0 03 Jun 2021
DynaEval: Unifying Turn and Dialogue Level Evaluation Chen Zhang Yiming Chen L. F. D’Haro Yan Zhang Thomas Friedrichs Grandee Lee Haizhou Li 19 72 0 02 Jun 2021
Improving Formality Style Transfer with Context-Aware Rule Injection Zonghai Yao Hong-ye Yu 18 16 0 01 Jun 2021
CLEVE: Contrastive Pre-training for Event Extraction Ziqi Wang Xiaozhi Wang Xu Han Yankai Lin Lei Hou Zhiyuan Liu Peng Li Juan-Zi Li Jie Zhou 26 115 0 30 May 2021
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging Xiaotao Gu Zihan Wang Zhenyu Bi Yu Meng Liyuan Liu Jiawei Han Jingbo Shang 60 36 0 28 May 2021
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents Chaojun Xiao Xueyu Hu Zhiyuan Liu Cunchao Tu Maosong Sun AILaw ELM 26 229 0 09 May 2021
Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge Yong Dai Jian-Dong Liu Jian Zhang H. Fu Zenglin Xu 17 12 0 09 May 2021
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality Adithya V Ganesan Matthew Matero Aravind Reddy Ravula Huy-Hien Vu H. A. Schwartz 17 35 0 07 May 2021
Framing Unpacked: A Semi-Supervised Interpretable Multi-View Model of Media Frames Shima Khanehzar Trevor Cohn Gosia Mikołajczak A. Turpin Lea Frermann 22 11 0 22 Apr 2021
Reference-based Weak Supervision for Answer Sentence Selection using Web Data Vivek Krishnamurthy Thuy Vu Alessandro Moschitti 11 1 0 18 Apr 2021
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts Arie Cattan Sophie Johnson Daniel S. Weld Ido Dagan Iz Beltagy Doug Downey Tom Hope 20 23 0 18 Apr 2021
Sequential Cross-Document Coreference Resolution Emily Allaway Shuai Wang Miguel Ballesteros 16 16 0 17 Apr 2021
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning Katerina Margatina Loïc Barrault Nikolaos Aletras 19 36 0 16 Apr 2021
Capturing Row and Column Semantics in Transformer Based Question Answering over Tables Michael R. Glass Mustafa Canim A. Gliozzo Saneem A. Chemmengath Vishwajeet Kumar Rishav Chakravarti Avirup Sil FeiFei Pan Samarth Bharadwaj Nicolas Rodolfo Fauceglia LMTD 11 54 0 16 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 18 164 0 16 Apr 2021
Towards Robust Neural Retrieval Models with Synthetic Pre-Training Revanth Reddy Gangi Reddy Vikas Yadav Md Arafat Sultan M. Franz Vittorio Castelli Heng Ji Avirup Sil 16 14 0 15 Apr 2021
Detoxifying Language Models Risks Marginalizing Minority Voices Albert Xu Eshaan Pathak Eric Wallace Suchin Gururangan Maarten Sap Dan Klein 13 121 0 13 Apr 2021
Self-Supervised Pretraining Improves Self-Supervised Pretraining Colorado Reed Xiangyu Yue Aniruddha Nrusimha Sayna Ebrahimi Vivek Vijaykumar ... Shanghang Zhang Devin Guillory Sean L. Metzger Kurt Keutzer Trevor Darrell 25 105 0 23 Mar 2021
Improving and Simplifying Pattern Exploiting Training Derek Tam Rakesh R Menon Mohit Bansal Shashank Srivastava Colin Raffel 13 149 0 22 Mar 2021
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review Dan Hendrycks Collin Burns Anya Chen Spencer Ball ELM AILaw 18 179 0 10 Mar 2021
Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do P. Schramowski Cigdem Turan Nico Andersen Constantin Rothkopf Kristian Kersting 25 281 0 08 Mar 2021
"Sharks are not the threat humans are": Argument Component Segmentation in School Student Essays Tariq Alhindi Debanjan Ghosh 11 12 0 08 Mar 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang D. Song Jacob Steinhardt ReLM FaML 55 1,804 0 05 Mar 2021
Gradual Fine-Tuning for Low-Resource Domain Adaptation Haoran Xu Seth Ebner M. Yarmohammadi A. White Benjamin Van Durme Kenton W. Murray CLL 14 39 0 03 Mar 2021
ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning Xin Xie Xiangnan Chen Xiang Chen Yong Wang Ningyu Zhang Shumin Deng Huajun Chen 34 2 0 25 Feb 2021
BERT-based Acronym Disambiguation with Multiple Training Strategies Chunguang Pan Bingyan Song Shengguang Wang Zhipeng Luo 17 18 0 25 Feb 2021
LogME: Practical Assessment of Pre-trained Models for Transfer Learning Kaichao You Yong Liu Jianmin Wang Mingsheng Long 16 178 0 22 Feb 2021
Characterizing English Variation across Social Media Communities with BERT L. Lucy David Bamman 16 35 0 12 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 30 207 0 03 Feb 2021
Word Alignment by Fine-tuning Embeddings on Parallel Corpora Zi-Yi Dou Graham Neubig 96 257 0 20 Jan 2021
Task Adaptive Pretraining of Transformers for Hostility Detection Tathagata Raha Sayar Ghosh Roy Ujwal Narayan Zubair Abid Vasudeva Varma 11 9 0 09 Jan 2021
Studying Strategically: Learning to Mask for Closed-book QA Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa OffRL 19 11 0 31 Dec 2020