RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,476 papers shown

Title
CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations Fuli Luo Pengcheng Yang Shicheng Li Xuancheng Ren Xu Sun VLM SSL 13 16 0 13 Oct 2020
Humane Visual AI: Telling the Stories Behind a Medical Condition Wonyoung So Edyta P. Bogucka S. Šćepanović Sagar Joglekar Ke Zhou Daniele Quercia 14 13 0 13 Oct 2020
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang Antonios Anastasopoulos Jun Araki Haibo Ding Graham Neubig HILM KELM 13 136 0 13 Oct 2020
Are Some Words Worth More than Others? Shiran Dudy Steven Bedrick 13 14 0 12 Oct 2020
Webly Supervised Image Classification with Metadata: Automatic Noisy Label Correction via Visual-Semantic Graph Jingkang Yang Weirong Chen Litong Feng Xiaopeng Yan Huabin Zheng Wayne Zhang NoLa 25 13 0 12 Oct 2020
Reformulating Unsupervised Style Transfer as Paraphrase Generation Kalpesh Krishna John Wieting Mohit Iyyer 19 237 0 12 Oct 2020
Counterfactual Variable Control for Robust and Interpretable Question Answering S. Yu Yulei Niu Shuohang Wang Jing Jiang Qianru Sun AAML OOD 40 9 0 12 Oct 2020
Neural, Symbolic and Neural-Symbolic Reasoning on Knowledge Graphs Jing Zhang Bo Chen Lingxi Zhang Xirui Ke Haipeng Ding NAI 23 3 0 12 Oct 2020
A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies Ho-Lam Chung Ying-Hong Chan Yao-Chung Fan 31 41 0 12 Oct 2020
Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis Roy Bar-Haim Yoav Kantor Lilach Eden Roni Friedman Dan Lahav Noam Slonim 24 43 0 11 Oct 2020
Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make It Danielle Saunders Rosie Sallis Bill Byrne 11 63 0 11 Oct 2020
SMYRF: Efficient Attention using Asymmetric Clustering Giannis Daras Nikita Kitaev Augustus Odena A. Dimakis 23 44 0 11 Oct 2020
SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task Z. Li Hai Zhao Rui Wang Kehai Chen Masao Utiyama Eiichiro Sumita 29 15 0 11 Oct 2020
Automated Concatenation of Embeddings for Structured Prediction Xinyu Wang Yong-jia Jiang Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 35 172 0 10 Oct 2020
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data William Huang Haokun Liu Samuel R. Bowman 13 37 0 09 Oct 2020
Precise Task Formalization Matters in Winograd Schema Evaluations Haokun Liu William Huang Dhara Mungra Samuel R. Bowman ReLM 17 12 0 08 Oct 2020
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders Jue Wang Wei Lu 15 224 0 08 Oct 2020
Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition Yun He Ziwei Zhu Yin Zhang Qin Chen James Caverlee AI4MH 28 108 0 08 Oct 2020
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing Xilun Chen Asish Ghoshal Yashar Mehdad Luke Zettlemoyer S. Gupta 22 89 0 07 Oct 2020
Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision Max Glockner Ivan Habernal Iryna Gurevych LRM 14 25 0 07 Oct 2020
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction M. Chen Tao Ge Xingxing Zhang Furu Wei M. Zhou 6 46 0 07 Oct 2020
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective Boxin Wang Shuohang Wang Yu Cheng Zhe Gan R. Jia Bo-wen Li Jingjing Liu AAML 38 113 0 05 Oct 2020
PMI-Masking: Principled masking of correlated spans Yoav Levine Barak Lenz Opher Lieber Omri Abend Kevin Leyton-Brown Moshe Tennenholtz Y. Shoham 11 72 0 05 Oct 2020
How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers? Shayne Longpre Yu Wang Christopher DuBois ViT 17 83 0 05 Oct 2020
Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models Thuy-Trang Vu Dinh Q. Phung Gholamreza Haffari 6 24 0 05 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 6 33 0 04 Oct 2020
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels Ilias Chalkidis Manos Fergadiotis Sotiris Kotitsas Prodromos Malakasiotis Nikolaos Aletras Ion Androutsopoulos VLM AI4TS 10 84 0 04 Oct 2020
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space Dayiheng Liu Yeyun Gong Jie Fu Yu Yan Jiusheng Chen Jiancheng Lv Nan Duan M. Zhou 10 37 0 04 Oct 2020
Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media Xiang Dai Sarvnaz Karimi Ben Hachey Cécile Paris 11 35 0 02 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 22 662 0 02 Oct 2020
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale Andreas Rucklé Jonas Pfeiffer Iryna Gurevych 14 37 0 02 Oct 2020
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling Yan Shvartzshnaider Ananth Balashankar Vikas Patidar Thomas Wies L. Subramanian 19 4 0 01 Oct 2020
CoLAKE: Contextualized Language and Knowledge Embedding Tianxiang Sun Yunfan Shao Xipeng Qiu Qipeng Guo Yaru Hu Xuanjing Huang Zheng-Wei Zhang KELM 18 181 0 01 Oct 2020
Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting Anshul Wadhawan 14 7 0 01 Oct 2020
RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble Michael Bendersky Honglei Zhuang Ji Ma Shuguang Han Keith B. Hall Ryan T. McDonald 19 16 0 01 Oct 2020
Examining the rhetorical capacities of neural language models Zining Zhu Chuer Pan Mohamed Abdalla Frank Rudzicz 28 10 0 01 Oct 2020
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 6 641 0 30 Sep 2020
Bridging Information-Seeking Human Gaze and Machine Reading Comprehension J. Malmaud R. Levy Yevgeni Berzak 14 31 0 30 Sep 2020
Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learning Subhojeet Pramanik Shashank Mujumdar Hima Patel 11 31 0 30 Sep 2020
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank Ethan C. Chau Lucy H. Lin Noah A. Smith 19 15 0 29 Sep 2020
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Tao Yu Chien-Sheng Wu Xi Victoria Lin Bailin Wang Y. Tan Xinyi Yang Dragomir R. Radev R. Socher Caiming Xiong LMTD 19 247 0 29 Sep 2020
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation Dinghan Shen Ming Zheng Yelong Shen Yanru Qu Weizhu Chen AAML 21 130 0 29 Sep 2020
Double Graph Based Reasoning for Document-level Relation Extraction Shuang Zeng Runxin Xu Baobao Chang Lei Li 8 223 0 29 Sep 2020
Conversational Semantic Parsing Armen Aghajanyan Jean Maillard Akshat Shrivastava K. Diedrick Mike Haeger ... Yashar Mehdad Ves Stoyanov Anuj Kumar M. Lewis S. Gupta 11 48 0 28 Sep 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 21 188 0 26 Sep 2020
Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling Arie Cattan Alon Eirew Gabriel Stanovsky Mandar Joshi Ido Dagan 11 35 0 23 Sep 2020
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 30 429 0 22 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 19 89 0 19 Sep 2020
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks Trapit Bansal Rishikesh Jha Tsendsuren Munkhdalai Andrew McCallum SSL VLM 20 87 0 17 Sep 2020
A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support Ashish Sharma Adam S. Miner David C. Atkins Tim Althoff AI4MH 25 268 0 17 Sep 2020