RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,743 papers shown

Title
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 62 2,618 0 05 Jun 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 42 229 0 05 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 24 72 0 31 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 15 39,979 0 28 May 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport Kyle Swanson L. Yu Tao Lei OT 24 37 0 27 May 2020
GECToR -- Grammatical Error Correction: Tag, Not Rewrite Kostiantyn Omelianchuk Vitaliy Atrasevych Artem Chernodub Oleksandr Skurzhanskyi 22 304 0 26 May 2020
Sentiment Analysis: Automatically Detecting Valence, Emotions, and Other Affectual States from Text Saif M. Mohammad 25 312 0 25 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavavs KELM 31 79 0 24 May 2020
L2R2: Leveraging Ranking for Abductive Reasoning Yunchang Zhu Liang Pang Yanyan Lan Xueqi Cheng 11 14 0 22 May 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Vu A. Nguyen VLM 32 900 0 20 May 2020
SciSight: Combining faceted navigation and research group detection for COVID-19 exploratory scientific search Tom Hope Jason Portenoy Kishore Vasan Jon Borchardt Eric Horvitz Daniel S. Weld Marti A. Hearst Jevin D. West FedML 13 58 0 20 May 2020
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs Yongkweon Jeon Baeseong Park S. Kwon Byeongwook Kim Jeongin Yun Dongsoo Lee MQ 25 30 0 20 May 2020
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text Felix Hill Soňa Mokrá Nathaniel Wong Tim Harley LM&Ro 13 81 0 19 May 2020
Table Search Using a Deep Contextualized Language Model Zhiyu Zoey Chen M. Trabelsi J. Heflin Yinan Xu Brian D. Davison LMTD 18 56 0 19 May 2020
Quantifying the Uncertainty of Precision Estimates for Rule based Text Classifiers J. Nutaro Özgür Özmen 16 0 0 19 May 2020
Are All Languages Created Equal in Multilingual BERT? Shijie Wu Mark Dredze 8 316 0 18 May 2020
Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations Sam Coope Tyler Farghly D. Gerz Ivan Vulić Matthew Henderson 19 62 0 18 May 2020
T-VSE: Transformer-Based Visual Semantic Embedding M. Bastan Arnau Ramisa Mehmet Tek ViT 16 7 0 17 May 2020
ApplicaAI at SemEval-2020 Task 11: On RoBERTa-CRF, Span CLS and Whether Self-Training Helps Them Dawid Jurkiewicz Łukasz Borchmann Izabela Kosmala Filip Graliñski 14 39 0 16 May 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh Thomas Wolf Alexander M. Rush 19 466 0 15 May 2020
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter Martin Müller M. Salathé P. Kummervold VLM MedIm AI4MH 24 355 0 15 May 2020
That is a Known Lie: Detecting Previously Fact-Checked Claims Shaden Shaar Giovanni Da San Martino Nikolay Babulkov Preslav Nakov HILM 44 153 0 12 May 2020
A Report on the 2020 Sarcasm Detection Shared Task Debanjan Ghosh Avijit Vajpayee Smaranda Muresan 16 59 0 12 May 2020
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge Hongming Zhang Xinran Zhao Yangqiu Song 16 54 0 12 May 2020
On the Robustness of Language Encoders against Grammatical Errors Fan Yin Quanyu Long Tao Meng Kai-Wei Chang 31 34 0 12 May 2020
SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching Baolin Peng Chunyuan Li Jinchao Li Shahin Shayandeh Lars Liden Jianfeng Gao 25 125 0 11 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 25 14 0 11 May 2020
schuBERT: Optimizing Elements of BERT A. Khetan Zohar S. Karnin 23 30 0 09 May 2020
Temporal Common Sense Acquisition with Minimal Supervision Ben Zhou Qiang Ning Daniel Khashabi Dan Roth 19 92 0 08 May 2020
Evidence Inference 2.0: More Data, Better Models Jay DeYoung Eric P. Lehman Benjamin E. Nye Iain J. Marshall Byron C. Wallace 17 68 0 08 May 2020
Beyond Accuracy: Behavioral Testing of NLP models with CheckList Marco Tulio Ribeiro Tongshuang Wu Carlos Guestrin Sameer Singh ELM 6 1,080 0 08 May 2020
Detecting East Asian Prejudice on Social Media Bertie Vidgen Austin Botelho David A. Broniatowski E. Guest Matthew Hall Helen Z. Margetts Rebekah Tromble Zeerak Talat Scott A. Hale 6 97 0 08 May 2020
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference Ali Hadi Zadeh Isak Edo Omar Mohamed Awad Andreas Moshovos MQ 22 183 0 08 May 2020
Blind Backdoors in Deep Learning Models Eugene Bagdasaryan Vitaly Shmatikov AAML FedML SILM 24 298 0 08 May 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei-Ye Zhao Steffen Eger ELM 19 124 0 07 May 2020
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders Terra Blevins Luke Zettlemoyer 19 162 0 06 May 2020
The Cascade Transformer: an Application for Efficient Answer Sentence Selection Luca Soldaini Alessandro Moschitti 19 44 0 05 May 2020
To Test Machine Comprehension, Start by Defining Comprehension Jesse Dunietz Greg Burnham Akash Bharadwaj Owen Rambow Jennifer Chu-Carroll D. Ferrucci FaML 54 65 0 04 May 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 59 34 0 04 May 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 27 161 0 03 May 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 18 154 0 02 May 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 19 36 0 02 May 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 19 68 0 02 May 2020
UnifiedQA: Crossing Format Boundaries With a Single QA System Daniel Khashabi Sewon Min Tushar Khot Ashish Sabharwal Oyvind Tafjord Peter Clark Hannaneh Hajishirzi 35 719 0 02 May 2020
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering Qingqing Cao H. Trivedi A. Balasubramanian Niranjan Balasubramanian 32 66 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 22 14 0 01 May 2020
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research Soujanya Poria Devamanyu Hazarika Navonil Majumder Rada Mihalcea 40 207 0 01 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 21 816 0 01 May 2020