RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,487 papers shown

Title
A Report on the 2020 Sarcasm Detection Shared Task Debanjan Ghosh Avijit Vajpayee Smaranda Muresan 16 59 0 12 May 2020
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge Hongming Zhang Xinran Zhao Yangqiu Song 16 54 0 12 May 2020
On the Robustness of Language Encoders against Grammatical Errors Fan Yin Quanyu Long Tao Meng Kai-Wei Chang 31 33 0 12 May 2020
SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching Baolin Peng Chunyuan Li Jinchao Li Shahin Shayandeh Lars Liden Jianfeng Gao 25 125 0 11 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 25 14 0 11 May 2020
schuBERT: Optimizing Elements of BERT A. Khetan Zohar S. Karnin 23 30 0 09 May 2020
Temporal Common Sense Acquisition with Minimal Supervision Ben Zhou Qiang Ning Daniel Khashabi Dan Roth 19 92 0 08 May 2020
Evidence Inference 2.0: More Data, Better Models Jay DeYoung Eric P. Lehman Benjamin E. Nye Iain J. Marshall Byron C. Wallace 11 68 0 08 May 2020
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference Ali Hadi Zadeh Isak Edo Omar Mohamed Awad Andreas Moshovos MQ 22 183 0 08 May 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei-Ye Zhao Steffen Eger ELM 11 124 0 07 May 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 52 64 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 54 34 0 04 May 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 22 161 0 03 May 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 18 154 0 02 May 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 19 36 0 02 May 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 19 68 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 22 14 0 01 May 2020
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea 40 207 0 01 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 41 492 0 01 May 2020
Elastic weight consolidation for better bias inoculation James Thorne Andreas Vlachos 17 11 0 29 Apr 2020
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning Alexandre Tamborrino Nicola Pellicanò B. Pannier Pascal Voitot Louise Naudin LRM 14 62 0 29 Apr 2020
Span-based Localizing Network for Natural Language Video Localization Hao Zhang Aixin Sun Wei Jing Joey Tianyi Zhou 15 311 0 29 Apr 2020
Revisiting Pre-Trained Models for Chinese Natural Language Processing Yiming Cui Wanxiang Che Ting Liu Bing Qin Shijin Wang Guoping Hu 26 681 0 29 Apr 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference Ji Xin Raphael Tang Jaejun Lee Yaoliang Yu Jimmy J. Lin 6 363 0 27 Apr 2020
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering Alexander R. Fabbri Patrick K. L. Ng Zhiguo Wang Ramesh Nallapati Bing Xiang 24 77 0 24 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 22 91 0 24 Apr 2020
QURIOUS: Question Generation Pretraining for Text Generation Shashi Narayan Gonçalo Simães Ji Ma Hannah Craighead Ryan T. McDonald 26 15 0 23 Apr 2020
A Review of Winograd Schema Challenge Datasets and Approaches Vid Kocijan Thomas Lukasiewicz E. Davis G. Marcus L. Morgenstern 25 43 0 23 Apr 2020
Residual Energy-Based Models for Text Generation Yuntian Deng A. Bakhtin Myle Ott Arthur Szlam MarcÁurelio Ranzato 20 125 0 22 Apr 2020
The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic Viet-An Duong Phu Pham Tongyu Yang Yu Wang Jiebo Luo AI4CE 14 90 0 21 Apr 2020
Train No Evil: Selective Masking for Task-Guided Pre-Training Yuxian Gu Zhengyan Zhang Xiaozhi Wang Zhiyuan Liu Maosong Sun 24 59 0 21 Apr 2020
StereoSet: Measuring stereotypical bias in pretrained language models Moin Nadeem Anna Bethke Siva Reddy 20 954 0 20 Apr 2020
Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network Guanming Xiong 21 0 0 20 Apr 2020
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings Masoud Jalili Sabet Philipp Dufter François Yvon Hinrich Schütze 23 226 0 18 Apr 2020
CLUE: A Chinese Language Understanding Evaluation Benchmark Liang Xu Hai Hu Xuanwei Zhang Lu Li Chenjie Cao ... Cong Yue Xinrui Zhang Zhen-Yi Yang Kyle Richardson Zhenzhong Lan ELM 31 377 0 13 Apr 2020
From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap Shuyang Gao Sanchit Agarwal Tagyoung Chung Di Jin Dilek Z. Hakkani-Tür 18 71 0 13 Apr 2020
Unsupervised Commonsense Question Answering with Self-Talk Vered Shwartz Peter West Ronan Le Bras Chandra Bhagavatula Yejin Choi ReLM SSL AI4MH LRM 14 257 0 11 Apr 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 28 70 0 10 Apr 2020
Translation Artifacts in Cross-lingual Transfer Learning Mikel Artetxe Gorka Labaka Eneko Agirre 19 114 0 09 Apr 2020
BLEURT: Learning Robust Metrics for Text Generation Thibault Sellam Dipanjan Das Ankur P. Parikh 46 1,439 0 09 Apr 2020
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 26 155 0 08 Apr 2020
Downstream Model Design of Pre-trained Language Model for Relation Extraction Task Cheng-rong Li Ye Tian 11 36 0 08 Apr 2020
DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement Tianda Li Jia-Chen Gu Xiao-Dan Zhu Quan Liu Zhenhua Ling Zhiming Su Si Wei 21 27 0 08 Apr 2020
Byte Pair Encoding is Suboptimal for Language Model Pretraining Kaj Bostrom Greg Durrett 14 200 0 07 Apr 2020
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases Donghyun Choi M. Shin EungGyun Kim Dong Ryeol Shin 23 123 0 07 Apr 2020
Evaluating Models' Local Decision Boundaries via Contrast Sets Matt Gardner Yoav Artzi Victoria Basmova Jonathan Berant Ben Bogin ... Sanjay Subramanian Reut Tsarfaty Eric Wallace Ally Zhang Ben Zhou ELM 35 84 0 06 Apr 2020
TAPAS: Weakly Supervised Table Parsing via Pre-training Jonathan Herzig Pawel Krzysztof Nowak Thomas Müller Francesco Piccinno Julian Martin Eisenschlos LMTD RALM 19 632 0 05 Apr 2020
FastBERT: a Self-distilling BERT with Adaptive Inference Time Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Haotang Deng Qi Ju 31 354 0 05 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 24 243 0 05 Apr 2020
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space Chunyuan Li Xiang Gao Yuan Li Baolin Peng Xiujun Li Yizhe Zhang Jianfeng Gao SSL DRL 30 181 0 05 Apr 2020