RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,476 papers shown

Title
GraphCodeBERT: Pre-training Code Representations with Data Flow Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang ... Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang M. Zhou 56 1,094 0 17 Sep 2020
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning Bingbing Li Zhenglun Kong Tianyun Zhang Ji Li Z. Li Hang Liu Caiwen Ding VLM 24 64 0 17 Sep 2020
Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches Juan Cruz-Benito Sanjay Vishwakarma Francisco Martín-Fernández Ismael Faro Ibm Quantum 22 30 0 16 Sep 2020
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow Li Zhang Qing Lyu Chris Callison-Burch ReLM LRM 11 85 0 16 Sep 2020
Critical Thinking for Language Models Gregor Betz Christian Voigt Kyle Richardson SyDa ReLM LRM AI4CE 18 35 0 15 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 22 953 0 15 Sep 2020
MLMLM: Link Prediction with Mean Likelihood Masked Language Model Louis Clouâtre P. Trempe Amal Zouaq Sarath Chandar 17 43 0 15 Sep 2020
GeDi: Generative Discriminator Guided Sequence Generation Ben Krause Akhilesh Deepak Gotmare Bryan McCann N. Keskar Shafiq R. Joty R. Socher Nazneen Rajani 51 389 0 14 Sep 2020
Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues Ruijian Xu Chongyang Tao Daxin Jiang Xueliang Zhao Dongyan Zhao Rui Yan 24 70 0 14 Sep 2020
On Robustness and Bias Analysis of BERT-based Relation Extraction Luoqiu Li Xiang Chen Hongbin Ye Zhen Bi Shumin Deng Ningyu Zhang Huajun Chen 24 18 0 14 Sep 2020
Cosine meets Softmax: A tough-to-beat baseline for visual grounding N. Rufus U. R. Nair K. M. Krishna Vineet Gandhi 22 13 0 13 Sep 2020
Differentially Private Language Models Benefit from Public Pre-training Gavin Kerrigan Dylan Slack Jens Tuyls 13 56 0 13 Sep 2020
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge Kai Sun Dian Yu Jianshu Chen Dong Yu Claire Cardie 25 12 0 12 Sep 2020
CIA_NITT at WNUT-2020 Task 2: Classification of COVID-19 Tweets Using Pre-trained Language Models Yandrapati Prakash Babu Eswari Rajagopal 19 9 0 12 Sep 2020
Sparsifying Transformer Models with Trainable Representation Pooling Michal Pietruszka Łukasz Borchmann Lukasz Garncarek 13 10 0 10 Sep 2020
Brain2Word: Decoding Brain Activity for Language Generation Nicolas Affolter Béni Egressy Damian Pascual Roger Wattenhofer 9 21 0 10 Sep 2020
Exploiting Multi-Modal Features From Pre-trained Networks for Alzheimer's Dementia Recognition Junghyun Koo Jie Hwan Lee Jaewoo Pyo Yujin Jo Kyogu Lee 11 58 0 09 Sep 2020
kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification Jiaxiang Liu Xuyi Chen Shikun Feng Shuohuan Wang Ouyang Xuan Yu Sun Zhengjie Huang Weiyue Su 27 19 0 08 Sep 2020
Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models Evan Williams Paul Rodrigues Valerie Novak 31 42 0 05 Sep 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 15 83 0 03 Sep 2020
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives Alexander Mathis Steffen Schneider Jessy Lauer Mackenzie W. Mathis 20 165 0 01 Sep 2020
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach William Falcon Kyunghyun Cho SSL 11 103 0 31 Aug 2020
A Survey of Evaluation Metrics Used for NLG Systems Ananya B. Sai Akash Kumar Mohankumar Mitesh M. Khapra ELM 25 228 0 27 Aug 2020
Analysis and Evaluation of Language Models for Word Sense Disambiguation Daniel Loureiro Kiamehr Rezaee Mohammad Taher Pilehvar Jose Camacho-Collados 10 13 0 26 Aug 2020
Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance Selim F. Yilmaz E. Kaynak Aykut Koç H. Dibeklioğlu Suleyman Serdar Kozat 29 26 0 26 Aug 2020
Conceptualized Representation Learning for Chinese Biomedical Text Mining Ningyu Zhang Qianghuai Jia Kangping Yin Liang Dong Feng Gao Nengwei Hua OOD 29 65 0 25 Aug 2020
How Have We Reacted To The COVID-19 Pandemic? Analyzing Changing Indian Emotions Through The Lens of Twitter Rajdeep Mukherjee S. Poddar Atharva Naik Soham Dasgupta 17 5 0 20 Aug 2020
Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems Andrea Madotto Zihan Liu Zhaojiang Lin Pascale Fung 38 58 0 14 Aug 2020
Hybrid Ranking Network for Text-to-SQL Qin Lyu K. Chakrabarti Shobhit Hathi Souvik Kundu Jianwen Zhang Zheng Chen AIMat 9 83 0 11 Aug 2020
KR-BERT: A Small-Scale Korean-Specific Language Model Sangah Lee Hansol Jang Yunmee Baik Suzi Park Hyopil Shin 14 51 0 10 Aug 2020
Distilling the Knowledge of BERT for Sequence-to-Sequence ASR Hayato Futami H. Inaguma Sei Ueno Masato Mimura S. Sakai Tatsuya Kawahara 19 50 0 09 Aug 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 32 156 0 06 Aug 2020
Aligning AI With Shared Human Values Dan Hendrycks Collin Burns Steven Basart Andrew Critch J. Li D. Song Jacob Steinhardt 32 515 0 05 Aug 2020
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning Y. Tang C. Tran Xian Li Peng-Jen Chen Naman Goyal Vishrav Chaudhary Jiatao Gu Angela Fan CLL 47 445 0 02 Aug 2020
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering Shayne Longpre Yi Lu Joachim Daiber ELM HILM 30 151 0 30 Jul 2020
Representation Learning with Video Deep InfoMax R. Devon Hjelm Philip Bachman SSL MDE 14 28 0 27 Jul 2020
Reed at SemEval-2020 Task 9: Fine-Tuning and Bag-of-Words Approaches to Code-Mixed Sentiment Analysis Vinay Gopalan Mark Hopkins 20 6 0 26 Jul 2020
Named entity recognition in chemical patents using ensemble of contextual language models J. Copara Nona Naderi J. Knafou Patrick Ruch Douglas Teodoro 21 23 0 24 Jul 2020
FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings Bertelt Braaksma R. Scholtens Stan van Suijlekom Remy Wang A. Ustun 15 3 0 24 Jul 2020
Clustering of Social Media Messages for Humanitarian Aid Response during Crisis Swati Padhee T. K. Saha Joel R. Tetreault A. Jaimes 17 6 0 23 Jul 2020
Investigating Pretrained Language Models for Graph-to-Text Generation Leonardo F. R. Ribeiro Martin Schmitt Hinrich Schütze Iryna Gurevych 17 215 0 16 Jul 2020
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning Jian Liu Leyang Cui Hanmeng Liu Dandan Huang Yile Wang Yue Zhang RALM 6 331 0 16 Jul 2020
Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms Firoj Alam Fahim Dalvi Shaden Shaar Nadir Durrani Hamdy Mubarak ... Giovanni Da San Martino Ahmed Abdelali Hassan Sajjad Kareem Darwish Preslav Nakov 14 102 0 15 Jul 2020
Deep learning models for representing out-of-vocabulary words Johannes V. Lochter Renato M. Silva Tiago A. Almeida 11 15 0 14 Jul 2020
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu Garima Lalwani Spandana Gella He He LRM 19 184 0 14 Jul 2020
Learning Reasoning Strategies in End-to-End Differentiable Proving Pasquale Minervini Sebastian Riedel Pontus Stenetorp Edward Grefenstette Tim Rocktaschel LRM 37 96 0 13 Jul 2020
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech Andy T. Liu Shang-Wen Li Hung-yi Lee SSL 48 356 0 12 Jul 2020
Learning Sparse Prototypes for Text Generation Junxian He Taylor Berg-Kirkpatrick Graham Neubig 16 23 0 29 Jun 2020
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models Viet The Bui Oanh T. K. Tran Hong Phuong Le 17 38 0 29 Jun 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 19 376 0 26 Jun 2020