RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,487 papers shown

Title
Enhancing Factual Consistency of Abstractive Summarization Chenguang Zhu William Fu-Hinthorn Ruochen Xu Qingkai Zeng Michael Zeng Xuedong Huang Meng-Long Jiang HILM KELM 185 40 0 19 Mar 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,450 0 18 Mar 2020
Transformer Networks for Trajectory Forecasting Francesco Giuliari Irtiza Hasan Marco Cristani Fabio Galasso 113 371 0 18 Mar 2020
Learning to Encode Position for Transformer with Continuous Dynamical Model Xuanqing Liu Hsiang-Fu Yu Inderjit Dhillon Cho-Jui Hsieh 8 107 0 13 Mar 2020
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning Zhiyuan Fang Tejas Gokhale Pratyay Banerjee Chitta Baral Yezhou Yang 10 60 0 11 Mar 2020
Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT Aitor García-Pablos Naiara Pérez Montse Cuadros 29 34 0 06 Mar 2020
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference Tianyu Liu Xin Zheng Baobao Chang Zhifang Sui 40 23 0 05 Mar 2020
Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout Filip Graliñski Tomasz Stanislawek Anna Wróblewska Dawid Lipiñski Agnieszka Kaliska Paulina Rosalska Bartosz Topolski P. Biecek 23 40 0 04 Mar 2020
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection Daniele Bonadiman Alessandro Moschitti RALM 11 10 0 04 Mar 2020
Learning Representations by Predicting Bags of Visual Words Spyros Gidaris Andrei Bursuc N. Komodakis P. Pérez Matthieu Cord SSL 28 117 0 27 Feb 2020
On Feature Normalization and Data Augmentation Boyi Li Felix Wu Ser-Nam Lim Serge J. Belongie Kilian Q. Weinberger 13 134 0 25 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 45 1,198 0 25 Feb 2020
Training Question Answering Models From Synthetic Data Raul Puri Ryan Spring M. Patwary M. Shoeybi Bryan Catanzaro ELM 24 159 0 22 Feb 2020
From English To Foreign Languages: Transferring Pre-trained Language Models Ke M. Tran 22 47 0 18 Feb 2020
Robustness Verification for Transformers Zhouxing Shi Huan Zhang Kai-Wei Chang Minlie Huang Cho-Jui Hsieh AAML 19 104 0 16 Feb 2020
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks Carlos Aspillaga Andrés Carvallo Vladimir Araujo ELM 39 31 0 14 Feb 2020
FQuAD: French Question Answering Dataset Martin d'Hoffschmidt Wacim Belblidia Tom Brendlé Quentin Heinrich Maxime Vidal 21 98 0 14 Feb 2020
Feature Importance Estimation with Self-Attention Networks Blaž Škrlj S. Džeroski Nada Lavrac Matej Petković FAtt MILM 26 51 0 11 Feb 2020
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning Weihao Yu Zihang Jiang Yanfei Dong Jiashi Feng LRM 8 239 0 11 Feb 2020
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 34 220 0 10 Feb 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 13 1,987 0 10 Feb 2020
Pre-training Tasks for Embedding-based Large-scale Retrieval Wei-Cheng Chang Felix X. Yu Yin-Wen Chang Yiming Yang Sanjiv Kumar RALM 11 301 0 10 Feb 2020
Segmented Graph-Bert for Graph Instance Modeling Jiawei Zhang SSeg 25 6 0 09 Feb 2020
MA-DST: Multi-Attention Based Scalable Dialog State Tracking Adarsh Kumar Peter Ku Anuj Kumar Goyal A. Metallinou Dilek Z. Hakkani-Tür 6 58 0 07 Feb 2020
perm2vec: Graph Permutation Selection for Decoding of Error Correction Codes using Self-Attention Nir Raviv Avi Caciularu Tomer Raviv Jacob Goldberger Yair Be’ery 13 8 0 06 Feb 2020
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension Max Bartolo A. Roberts Johannes Welbl Sebastian Riedel Pontus Stenetorp AAML 13 167 0 02 Feb 2020
Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction Taeuk Kim Jihun Choi Daniel Edmiston Sang-goo Lee 22 90 0 30 Jan 2020
Retrospective Reader for Machine Reading Comprehension Zhuosheng Zhang Junjie Yang Hai Zhao RALM 23 226 0 27 Jan 2020
Generating Representative Headlines for News Stories Xiaotao Gu Yuning Mao Jiawei Han Jialu Liu Hongkun Yu You Wu Cong Yu Daniel Finnie Jiaqi Zhai Nicholas Zukoski 22 70 0 26 Jan 2020
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval Jie Lei Licheng Yu Tamara L. Berg Mohit Bansal 108 275 0 24 Jan 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 17 1,768 0 22 Jan 2020
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data Di Qi Lin Su Jianwei Song Edward Cui Taroon Bharti Arun Sacheti VLM 29 258 0 22 Jan 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,587 0 21 Jan 2020
RobBERT: a Dutch RoBERTa-based Language Model Pieter Delobelle Thomas Winters Bettina Berendt 10 232 0 17 Jan 2020
CLUENER2020: Fine-grained Named Entity Recognition Dataset and Benchmark for Chinese Liang Xu Yu Tong Qianqian Dong Yixuan Liao Cong Yu Yin Tian Weitang Liu Lu Li Caiquan Liu Xuanwei Zhang 30 48 0 13 Jan 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 17 300 0 31 Dec 2019
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 36 201 0 20 Dec 2019
BERTje: A Dutch BERT Model Wietse de Vries Andreas van Cranenburgh Arianna Bisazza Tommaso Caselli Gertjan van Noord Malvina Nissim VLM SSeg 11 291 0 19 Dec 2019
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 25 334 0 17 Dec 2019
Multilingual is not enough: BERT for Finnish Antti Virtanen Jenna Kanerva Rami Ilo Jouni Luoma Juhani Luotolahti T. Salakoski Filip Ginter S. Pyysalo 19 277 0 15 Dec 2019
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 35 395 0 11 Dec 2019
Sato: Contextual Semantic Type Detection in Tables Dan Zhang Yoshihiko Suhara Jinfeng Li Madelon Hulsebos cCaugatay Demiralp W. Tan LMTD 14 15 0 14 Nov 2019
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model Timothee Mickus Denis Paperno Mathieu Constant Kees van Deemter 16 45 0 13 Nov 2019
Adapting and evaluating a deep learning language model for clinical why-question answering Andrew Wen Mohamed Y. Elwazir Sungrim Moon Jungwei Fan LM&MA 11 31 0 13 Nov 2019
Neural Duplicate Question Detection without Labeled Training Data Andreas Rucklé N. Moosavi Iryna Gurevych OOD AAML 11 11 0 13 Nov 2019
Attending to Entities for Better Text Understanding Pengxiang Cheng K. Erk LRM 19 37 0 11 Nov 2019
How Decoding Strategies Affect the Verifiability of Generated Text Luca Massarelli Fabio Petroni Aleksandra Piktus Myle Ott Tim Rocktaschel Vassilis Plachouras Fabrizio Silvestri Sebastian Riedel 21 50 0 09 Nov 2019
What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning Jaejun Lee Raphael Tang Jimmy J. Lin 24 121 0 08 Nov 2019
Certified Data Removal from Machine Learning Models Chuan Guo Tom Goldstein Awni Y. Hannun L. V. D. van der Maaten MU 31 413 0 08 Nov 2019
S2ORC: The Semantic Scholar Open Research Corpus Kyle Lo Lucy Lu Wang Mark Neumann Rodney Michael Kinney Daniel S. Weld OffRL AI4CE 32 10 0 07 Nov 2019