RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 2,933 papers shown

Title
On the Computational Power of Transformers and its Implications in Sequence Modeling S. Bhattamishra Arkil Patel Navin Goyal 17 63 0 16 Jun 2020
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models Eyal Ben-David Carmel Rabinovitz Roi Reichart SSL 47 61 0 16 Jun 2020
Minimum Width for Universal Approximation Sejun Park Chulhee Yun Jaeho Lee Jinwoo Shin 19 121 0 16 Jun 2020
To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks Sinong Wang Madian Khabsa Hao Ma 8 26 0 15 Jun 2020
Self-supervised Learning: Generative or Contrastive Xiao Liu Fanjin Zhang Zhenyu Hou Zhaoyu Wang Li Mian Jing Zhang Jie Tang SSL 24 1,586 0 15 Jun 2020
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) Marcos Zampieri Preslav Nakov Sara Rosenthal Pepa Atanasova Georgi Karadzhov Hamdy Mubarak Leon Derczynski Zeses Pitenis cCaugri cColtekin 16 481 0 12 Jun 2020
VirTex: Learning Visual Representations from Textual Annotations Karan Desai Justin Johnson SSL VLM 19 432 0 11 Jun 2020
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP Libo Qin Minheng Ni Yue Zhang Wanxiang Che 22 149 0 11 Jun 2020
Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges M. Eskénazi Tiancheng Zhao LLMAG AI4TS AI4CE 30 9 0 10 Jun 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 30 441 0 10 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 22 1,643 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 27 98 0 05 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 62 2,614 0 05 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 20 72 0 31 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 15 39,839 0 28 May 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport Kyle Swanson L. Yu Tao Lei OT 24 37 0 27 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavavs KELM 21 79 0 24 May 2020
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs Yongkweon Jeon Baeseong Park S. Kwon Byeongwook Kim Jeongin Yun Dongsoo Lee MQ 22 30 0 20 May 2020
Table Search Using a Deep Contextualized Language Model Zhiyu Zoey Chen M. Trabelsi J. Heflin Yinan Xu Brian D. Davison LMTD 18 56 0 19 May 2020
Quantifying the Uncertainty of Precision Estimates for Rule based Text Classifiers J. Nutaro Özgür Özmen 11 0 0 19 May 2020
Are All Languages Created Equal in Multilingual BERT? Shijie Wu Mark Dredze 8 316 0 18 May 2020
Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations Sam Coope Tyler Farghly D. Gerz Ivan Vulić Matthew Henderson 16 62 0 18 May 2020
T-VSE: Transformer-Based Visual Semantic Embedding M. Bastan Arnau Ramisa Mehmet Tek ViT 13 7 0 17 May 2020
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter Martin Müller M. Salathé P. Kummervold VLM MedIm AI4MH 13 354 0 15 May 2020
That is a Known Lie: Detecting Previously Fact-Checked Claims Shaden Shaar Giovanni Da San Martino Nikolay Babulkov Preslav Nakov HILM 41 152 0 12 May 2020
A Report on the 2020 Sarcasm Detection Shared Task Debanjan Ghosh Avijit Vajpayee Smaranda Muresan 13 59 0 12 May 2020
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge Hongming Zhang Xinran Zhao Yangqiu Song 10 54 0 12 May 2020
On the Robustness of Language Encoders against Grammatical Errors Fan Yin Quanyu Long Tao Meng Kai-Wei Chang 25 33 0 12 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 25 14 0 11 May 2020
schuBERT: Optimizing Elements of BERT A. Khetan Zohar S. Karnin 15 30 0 09 May 2020
Temporal Common Sense Acquisition with Minimal Supervision Ben Zhou Qiang Ning Daniel Khashabi Dan Roth 16 92 0 08 May 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 52 64 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 54 34 0 04 May 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 22 162 0 03 May 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 10 154 0 02 May 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 16 36 0 02 May 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 13 68 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 22 14 0 01 May 2020
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea 37 207 0 01 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 41 491 0 01 May 2020
Elastic weight consolidation for better bias inoculation James Thorne Andreas Vlachos 14 11 0 29 Apr 2020
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning Alexandre Tamborrino Nicola Pellicanò B. Pannier Pascal Voitot Louise Naudin LRM 6 62 0 29 Apr 2020
Span-based Localizing Network for Natural Language Video Localization Hao Zhang Aixin Sun Wei Jing Joey Tianyi Zhou 15 311 0 29 Apr 2020
Revisiting Pre-Trained Models for Chinese Natural Language Processing Yiming Cui Wanxiang Che Ting Liu Bing Qin Shijin Wang Guoping Hu 20 682 0 29 Apr 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference Ji Xin Raphael Tang Jaejun Lee Yaoliang Yu Jimmy J. Lin 6 363 0 27 Apr 2020
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering Alexander R. Fabbri Patrick K. L. Ng Zhiguo Wang Ramesh Nallapati Bing Xiang 24 77 0 24 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 22 91 0 24 Apr 2020
QURIOUS: Question Generation Pretraining for Text Generation Shashi Narayan Gonçalo Simães Ji Ma Hannah Craighead Ryan T. McDonald 21 15 0 23 Apr 2020
A Review of Winograd Schema Challenge Datasets and Approaches Vid Kocijan Thomas Lukasiewicz E. Davis G. Marcus L. Morgenstern 12 43 0 23 Apr 2020