RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,476 papers shown

Title
skweak: Weak Supervision Made Easy for NLP Pierre Lison Jeremy Barnes A. Hubin 19 43 0 19 Apr 2021
Refining Targeted Syntactic Evaluation of Language Models Benjamin Newman Kai-Siang Ang Julia Gong John Hewitt 16 43 0 19 Apr 2021
BERTić -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian N. Ljubešić D. Lauc 8 48 0 19 Apr 2021
IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers Karthik Puranik Adeep Hande R. Priyadharshini Sajeetha Thavareesan Bharathi Raja Chakravarthi 15 59 0 19 Apr 2021
Natural Language Generation Using Link Grammar for General Conversational Intelligence Vignav Ramesh Anton Kolonin 11 2 0 19 Apr 2021
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation Rui Cheng Bichen Wu Peizhao Zhang Peter Vajda Joseph E. Gonzalez CLIP VLM 21 31 0 18 Apr 2021
Reference-based Weak Supervision for Answer Sentence Selection using Web Data Vivek Krishnamurthy Thuy Vu Alessandro Moschitti 11 1 0 18 Apr 2021
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts Arie Cattan Sophie Johnson Daniel S. Weld Ido Dagan Iz Beltagy Doug Downey Tom Hope 20 23 0 18 Apr 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 277 1,120 0 18 Apr 2021
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu Yizhe Zhang Chris Brockett Yi Mao Zhifang Sui Weizhu Chen W. Dolan HILM 217 143 0 18 Apr 2021
"Average" Approximates "First Principal Component"? An Empirical Analysis on Representations from Neural Language Models Zihan Wang Chengyu Dong Jingbo Shang FAtt 26 4 0 18 Apr 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 229 966 0 17 Apr 2021
The Topic Confusion Task: A Novel Scenario for Authorship Attribution Malik H. Altakrori Jackie C.K. Cheung Benjamin C. M. Fung 11 17 0 17 Apr 2021
Robust Embeddings Via Distributions Kira A. Selby Yinong Wang Ruizhe Wang Peyman Passban Ahmad Rashid Mehdi Rezagholizadeh Pascal Poupart OOD 19 3 0 17 Apr 2021
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning Katerina Margatina Loïc Barrault Nikolaos Aletras 19 36 0 16 Apr 2021
Membership Inference Attack Susceptibility of Clinical Language Models Abhyuday N. Jagannatha Bhanu Pratap Singh Rawat Hong-ye Yu MIACV 22 60 0 16 Apr 2021
AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha LM&MA MedIm 18 164 0 16 Apr 2021
Flexible Instance-Specific Rationalization of NLP Models G. Chrysostomou Nikolaos Aletras 23 14 0 16 Apr 2021
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema Yanai Elazar Hongming Zhang Yoav Goldberg Dan Roth ReLM LRM 37 44 0 16 Apr 2021
Supervising Model Attention with Human Explanations for Robust Natural Language Inference Joe Stacey Yonatan Belinkov Marek Rei 23 45 0 16 Apr 2021
Exploring Visual Engagement Signals for Representation Learning Menglin Jia Zuxuan Wu A. Reiter Claire Cardie Serge J. Belongie Ser-Nam Lim 19 13 0 15 Apr 2021
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data? Eric P. Lehman Sarthak Jain Karl Pichotta Yoav Goldberg Byron C. Wallace OOD MIACV 22 117 0 15 Apr 2021
How to Train BERT with an Academic Budget Peter Izsak Moshe Berchansky Omer Levy 12 112 0 15 Apr 2021
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 98 227 0 15 Apr 2021
Emotion Dynamics Modeling via BERT Haiqing Yang Jianping Shen 22 11 0 15 Apr 2021
K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce Song Xu Haoran Li Peng Yuan Yujia Wang Youzheng Wu Xiaodong He Ying Liu Bowen Zhou KELM 27 24 0 14 Apr 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 37 243 0 14 Apr 2021
AR-LSAT: Investigating Analytical Reasoning of Text Wanjun Zhong Siyuan Wang Duyu Tang Zenan Xu Daya Guo Jiahai Wang Jian Yin Ming Zhou Nan Duan ELM 19 40 0 14 Apr 2021
CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images Shailaja Keyur Sampat Akshay Kumar Yezhou Yang Chitta Baral 19 26 0 13 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 27 27 0 13 Apr 2021
Relational World Knowledge Representation in Contextual Language Models: A Review Tara Safavi Danai Koutra KELM 30 51 0 12 Apr 2021
Fighting the COVID-19 Infodemic with a Holistic BERT Ensemble Georgios Tziafas Konstantinos Kogkalidis Tommaso Caselli 16 9 0 12 Apr 2021
Escaping the Big Data Paradigm with Compact Transformers Ali Hassani Steven Walton Nikhil Shah Abulikemu Abuduweili Jiachen Li Humphrey Shi 54 462 0 12 Apr 2021
FUDGE: Controlled Text Generation With Future Discriminators Kevin Kaichuang Yang Dan Klein 13 313 0 12 Apr 2021
NLI Data Sanity Check: Assessing the Effect of Data Corruption on Model Performance Aarne Talman Marianna Apidianaki S. Chatzikyriakidis Jörg Tiedemann 20 10 0 10 Apr 2021
Deep Indexed Active Learning for Matching Heterogeneous Entity Representations Arjit Jain Sunita Sarawagi Prithviraj Sen 14 24 0 08 Apr 2021
COVID-19 Named Entity Recognition for Vietnamese Thinh Hung Truong M. Dao Dat Quoc Nguyen 21 38 0 08 Apr 2021
Layer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency Jinchuan Tian Rongzhi Gu Helin Wang Yuexian Zou 21 0 0 08 Apr 2021
HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks Firoj Alam U. Qazi Muhammad Imran Ferda Ofli 23 65 0 07 Apr 2021
Personalized Entity Resolution with Dynamic Heterogeneous Knowledge Graph Representations Ying Lin H. Wang Jiangning Chen Tong Wang Yue Liu Heng Ji Yang Liu Premkumar Natarajan 21 10 0 06 Apr 2021
What Will it Take to Fix Benchmarking in Natural Language Understanding? Samuel R. Bowman George E. Dahl ELM ALM 28 156 0 05 Apr 2021
Intent Detection and Slot Filling for Vietnamese M. Dao Thinh Hung Truong Dat Quoc Nguyen VLM 29 35 0 05 Apr 2021
A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles Sourya Dipta Das Ayan Basak S. Dutta 27 47 0 05 Apr 2021
Inference Time Style Control for Summarization Shuyang Cao Lu Wang AI4TS 22 15 0 05 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 23 986 0 31 Mar 2021
Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey Tapas Nayak Navonil Majumder Pawan Goyal Soujanya Poria ViT 12 49 0 31 Mar 2021
On the Adversarial Robustness of Vision Transformers Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen Cho-Jui Hsieh ViT 25 137 0 29 Mar 2021
Efficient Explanations from Empirical Explainers Robert Schwarzenberg Nils Feldhus Sebastian Möller FAtt 27 9 0 29 Mar 2021
Machine Learning Meets Natural Language Processing -- The story so far N. Galanis P. Vafiadis K.-G. Mirzaev G. Papakostas 30 6 0 27 Mar 2021
Bertinho: Galician BERT Representations David Vilares Marcos Garcia Carlos Gómez-Rodríguez 57 22 0 25 Mar 2021