Don't Stop Pretraining: Adapt Language Models to Domains and Tasks

23 April 2020

Kyle Lo

Papers citing "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"

50 / 352 papers shown

Title
Knowledge-Augmented Language Models for Cause-Effect Relation Classification Pedram Hosseini David A. Broniatowski Mona T. Diab CML 18 18 0 16 Dec 2021
Learning Rich Representation of Keyphrases from Text Mayank Kulkarni Debanjan Mahata Ravneet Arora Rajarshi Bhowmik VLM 19 65 0 16 Dec 2021
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval Kexin Wang Nandan Thakur Nils Reimers Iryna Gurevych VLM 14 149 0 14 Dec 2021
MoCA: Incorporating Multi-stage Domain Pretraining and Cross-guided Multimodal Attention for Textbook Question Answering Fangzhi Xu Qika Lin J. Liu Lingling Zhang Tianzhe Zhao Qianyi Chai Yudai Pan 9 2 0 06 Dec 2021
MultiVerS: Improving scientific claim verification with weak supervision and full-document context David Wadden Bertie Vidgen Lucy Lu Wang Dirk Hovy J. Pierrehumbert Hannaneh Hajishirzi 27 148 0 02 Dec 2021
NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging Zihan Liu Feijun Jiang Yuxiang Hu Chen Shi Pascale Fung 16 37 0 01 Dec 2021
Temporal Effects on Pre-trained Models for Language Processing Tasks Oshin Agarwal A. Nenkova VLM 14 52 0 24 Nov 2021
Merging Models with Fisher-Weighted Averaging Michael Matena Colin Raffel FedML MoMe 27 348 0 18 Nov 2021
Scaling Law for Recommendation Models: Towards General-purpose User Representations Kyuyong Shin Hanock Kwak KyungHyun Kim Max Nihlén Ramström Jisu Jeong Jung-Woo Ha S. Kim ELM 26 38 0 15 Nov 2021
SocialBERT -- Transformers for Online SocialNetwork Language Modelling I. Karpov Nick Kartashev 22 3 0 13 Nov 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 18 98 0 12 Nov 2021
Recent Advances in Automated Question Answering In Biomedical Domain K. D. Baksi 14 0 0 10 Nov 2021
Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks Wang Zhu Peter Shaw Tal Linzen Fei Sha 27 7 0 09 Nov 2021
Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks Aakanksha Naik J. Lehman Carolyn Rose 32 7 0 02 Nov 2021
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 26 15 0 26 Oct 2021
ClimateBert: A Pretrained Language Model for Climate-Related Text Nicolas Webersinke Mathias Kraus Jiabo Huang Markus Leippold AI4CE 21 131 0 22 Oct 2021
Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Prediction Shubhanshu Mishra A. Haghighi VLM 18 4 0 20 Oct 2021
Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition Li-Wei Chen Alexander I. Rudnicky VLM 8 121 0 12 Oct 2021
K-Wav2vec 2.0: Automatic Speech Recognition based on Joint Decoding of Graphemes and Syllables Jounghee Kim Pilsung Kang VLM 13 6 0 11 Oct 2021
Advances in Multi-turn Dialogue Comprehension: A Survey Zhuosheng Zhang Hai Zhao 21 21 0 11 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 21 36 0 09 Oct 2021
Improving Multi-Party Dialogue Discourse Parsing via Domain Integration Zhengyuan Liu Nancy F. Chen 16 33 0 09 Oct 2021
Machine Learning Featurizations for AI Hacking of Political Systems Nathan Sanders B. Schneier 18 2 0 08 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 222 150 0 07 Oct 2021
Rumour Detection via Zero-shot Cross-lingual Transfer Learning Lin Tian Xiuzhen Zhang Jey Han Lau 36 13 0 27 Sep 2021
Caption Enriched Samples for Improving Hateful Memes Detection Efrat Blaier Itzik Malkiel Lior Wolf VLM 51 21 0 22 Sep 2021
Cross-lingual Transfer of Monolingual Models Evangelia Gogoulou Ariel Ekgren T. Isbister Magnus Sahlgren 27 16 0 15 Sep 2021
Automatically Exposing Problems with Neural Dialog Models Dian Yu Kenji Sagae 18 9 0 14 Sep 2021
Learning Bill Similarity with Annotated and Augmented Corpora of Bills Jiseon Kim Elden Griggs In Song Kim Alice H. Oh AILaw 18 5 0 14 Sep 2021
Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding Shiyang Li Semih Yavuz Wenhu Chen Xifeng Yan 20 12 0 14 Sep 2021
IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization Fajri Koto Jey Han Lau Timothy Baldwin VLM 55 82 0 10 Sep 2021
Identifying Morality Frames in Political Tweets using Relational Learning Shamik Roy Maria Leonor Pacheco Dan Goldwasser 34 41 0 09 Sep 2021
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning Prasetya Ajie Utama N. Moosavi Victor Sanh Iryna Gurevych AAML 56 35 0 09 Sep 2021
Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense Wanyun Cui Xingran Chen 22 6 0 06 Sep 2021
Task-Oriented Dialogue System as Natural Language Generation Weizhi Wang Zhirui Zhang Junliang Guo Yinpei Dai Boxing Chen Weihua Luo 20 32 0 31 Aug 2021
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts Ashutosh Baheti Maarten Sap Alan Ritter Mark O. Riedl 16 84 0 26 Aug 2021
Small-Text: Active Learning for Text Classification in Python Christopher Schröder Lydia Muller A. Niekler Martin Potthast CLIP VLM AI4CE 31 23 0 21 Jul 2021
Adaptive Transfer Learning on Graph Neural Networks Xueting Han Zhenhuan Huang Bang An Jing Bai 22 59 0 19 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 26 17 0 04 Jul 2021
Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation Anne Lauscher Henning Wachsmuth Iryna Gurevych Goran Glavavs 25 31 0 01 Jul 2021
Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature Yu-Chiang Frank Wang Jinchao Li Tristan Naumann Chenyan Xiong Hao Cheng ... Yang Qin Eric Horvitz Paul N. Bennett Jianfeng Gao Hoifung Poon OOD 25 13 0 25 Jun 2021
GAIA: A Transfer Learning System of Object Detection that Fits Your Needs Xingyuan Bu Junran Peng Junjie Yan T. Tan Zhaoxiang Zhang ObjD VLM 20 53 0 21 Jun 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 16 221 0 18 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 25 27 0 16 Jun 2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data Haoming Jiang Danqing Zhang Tianyu Cao Bing Yin T. Zhao NoLa 19 44 0 16 Jun 2021
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 29 178 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 27 811 0 14 Jun 2021
CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing Sai Muralidhar Jayanthi Kavya Nerella Khyathi Raghavi Chandu A. Black MoE 23 8 0 10 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 29 11 0 10 Jun 2021
Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition Yihong Dong Ying Peng Muqiao Yang Songtao Lu Qingjiang Shi 38 9 0 05 Jun 2021