RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 2,766 papers shown

Title
Minimum Width for Universal Approximation Sejun Park Chulhee Yun Jaeho Lee Jinwoo Shin 17 121 0 16 Jun 2020
Self-supervised Learning: Generative or Contrastive Xiao Liu Fanjin Zhang Zhenyu Hou Zhaoyu Wang Li Mian Jing Zhang Jie Tang SSL 24 1,582 0 15 Jun 2020
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) Marcos Zampieri Preslav Nakov Sara Rosenthal Pepa Atanasova Georgi Karadzhov Hamdy Mubarak Leon Derczynski Zeses Pitenis cCaugri cColtekin 11 479 0 12 Jun 2020
VirTex: Learning Visual Representations from Textual Annotations Karan Desai Justin Johnson SSL VLM 14 432 0 11 Jun 2020
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP Libo Qin Minheng Ni Yue Zhang Wanxiang Che 12 148 0 11 Jun 2020
Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges M. Eskénazi Tiancheng Zhao LLMAG AI4TS AI4CE 28 9 0 10 Jun 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 21 438 0 10 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 17 1,637 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 17 98 0 05 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 62 2,607 0 05 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 20 71 0 31 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 10 39,839 0 28 May 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 21 33 0 27 May 2020
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport Kyle Swanson L. Yu Tao Lei OT 24 37 0 27 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavavs KELM 16 79 0 24 May 2020
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs Yongkweon Jeon Baeseong Park S. Kwon Byeongwook Kim Jeongin Yun Dongsoo Lee MQ 17 30 0 20 May 2020
Table Search Using a Deep Contextualized Language Model Zhiyu Zoey Chen M. Trabelsi J. Heflin Yinan Xu Brian D. Davison LMTD 10 56 0 19 May 2020
Quantifying the Uncertainty of Precision Estimates for Rule based Text Classifiers J. Nutaro Özgür Özmen 6 0 0 19 May 2020
Are All Languages Created Equal in Multilingual BERT? Shijie Wu Mark Dredze 6 316 0 18 May 2020
Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations Sam Coope Tyler Farghly D. Gerz Ivan Vulić Matthew Henderson 11 62 0 18 May 2020
T-VSE: Transformer-Based Visual Semantic Embedding M. Bastan Arnau Ramisa Mehmet Tek ViT 8 7 0 17 May 2020
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter Martin Müller M. Salathé P. Kummervold VLM MedIm AI4MH 8 354 0 15 May 2020
That is a Known Lie: Detecting Previously Fact-Checked Claims Shaden Shaar Giovanni Da San Martino Nikolay Babulkov Preslav Nakov HILM 36 152 0 12 May 2020
A Report on the 2020 Sarcasm Detection Shared Task Debanjan Ghosh Avijit Vajpayee Smaranda Muresan 11 59 0 12 May 2020
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge Hongming Zhang Xinran Zhao Yangqiu Song 8 54 0 12 May 2020
On the Robustness of Language Encoders against Grammatical Errors Fan Yin Quanyu Long Tao Meng Kai-Wei Chang 23 33 0 12 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 25 14 0 11 May 2020
schuBERT: Optimizing Elements of BERT A. Khetan Zohar S. Karnin 10 30 0 09 May 2020
Temporal Common Sense Acquisition with Minimal Supervision Ben Zhou Qiang Ning Daniel Khashabi Dan Roth 11 92 0 08 May 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 52 64 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 52 34 0 04 May 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 22 162 0 03 May 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 8 154 0 02 May 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 11 36 0 02 May 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 11 68 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 17 14 0 01 May 2020
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea 37 206 0 01 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 41 491 0 01 May 2020
Elastic weight consolidation for better bias inoculation James Thorne Andreas Vlachos 9 11 0 29 Apr 2020
Span-based Localizing Network for Natural Language Video Localization Hao Zhang Aixin Sun Wei Jing Joey Tianyi Zhou 10 311 0 29 Apr 2020
Revisiting Pre-Trained Models for Chinese Natural Language Processing Yiming Cui Wanxiang Che Ting Liu Bing Qin Shijin Wang Guoping Hu 12 681 0 29 Apr 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference Ji Xin Raphael Tang Jaejun Lee Yaoliang Yu Jimmy J. Lin 6 363 0 27 Apr 2020
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering Alexander R. Fabbri Patrick K. L. Ng Zhiguo Wang Ramesh Nallapati Bing Xiang 22 77 0 24 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 22 91 0 24 Apr 2020
QURIOUS: Question Generation Pretraining for Text Generation Shashi Narayan Gonçalo Simães Ji Ma Hannah Craighead Ryan T. McDonald 21 15 0 23 Apr 2020
Residual Energy-Based Models for Text Generation Yuntian Deng A. Bakhtin Myle Ott Arthur Szlam MarcÁurelio Ranzato 15 125 0 22 Apr 2020
The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic Viet-An Duong Phu Pham Tongyu Yang Yu Wang Jiebo Luo AI4CE 6 89 0 21 Apr 2020
Train No Evil: Selective Masking for Task-Guided Pre-Training Yuxian Gu Zhengyan Zhang Xiaozhi Wang Zhiyuan Liu Maosong Sun 14 59 0 21 Apr 2020
StereoSet: Measuring stereotypical bias in pretrained language models Moin Nadeem Anna Bethke Siva Reddy 6 952 0 20 Apr 2020
Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network Guanming Xiong 21 0 0 20 Apr 2020