RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

37 / 3,487 papers shown

Title
Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks Pavan Kapanipathi Veronika Thost S. Patel Spencer Whitehead Ibrahim Abdelaziz ... R. Chulaka Gunasekara B. Makni Nicholas Mattei Kartik Talamadupula Achille Fokoue 34 45 0 05 Nov 2019
Generalization through Memorization: Nearest Neighbor Language Models Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 48 807 0 01 Nov 2019
Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task Valeriya Slovikovskaya 11 41 0 31 Oct 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 41 30 0 29 Oct 2019
SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering Yung-Sung Chuang Chi-Liang Liu Hung-yi Lee Lin-shan Lee AuLLM 22 39 0 25 Oct 2019
HUBERT Untangles BERT to Improve Transfer across NLP Tasks M. Moradshahi Hamid Palangi M. Lam P. Smolensky Jianfeng Gao 21 16 0 25 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 71 19,415 0 23 Oct 2019
Generative Pre-Training for Speech with Autoregressive Predictive Coding Yu-An Chung James R. Glass SSL 15 173 0 23 Oct 2019
Improving Transformer-based Speech Recognition Using Unsupervised Pre-training Dongwei Jiang Xiaoning Lei Wubo Li Ne Luo Yuxuan Hu Wei Zou Xiangang Li 14 99 0 22 Oct 2019
Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection Giovanni Da San Martino Alberto Barrón-Cedeño Preslav Nakov 12 79 0 20 Oct 2019
Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models Tianxing He Jun Liu Kyunghyun Cho Myle Ott Bing-Quan Liu James R. Glass Fuchun Peng CLL 21 9 0 16 Oct 2019
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 44 70 0 09 Oct 2019
BERT for Evidence Retrieval and Claim Verification Shrishti Saha Shetu Christof Monz E. Mabande RALM 17 119 0 07 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 59 6,370 0 26 Sep 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 22 584 0 25 Sep 2019
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 237 205 0 25 Sep 2019
Portuguese Named Entity Recognition using BERT-CRF Fábio Souza Rodrigo Nogueira R. Lotufo 17 250 0 23 Sep 2019
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings Gregor Wiedemann Steffen Remus Avi Chawla Chris Biemann 16 174 0 23 Sep 2019
Language models and Automated Essay Scoring Pedro Uría Rodríguez Amir Jafari C. Ormerod 22 82 0 18 Sep 2019
Span-based Joint Entity and Relation Extraction with Transformer Pre-training Markus Eberts A. Ulges LRM ViT 164 380 0 17 Sep 2019
Frustratingly Easy Natural Question Answering Lin Pan Rishav Chakravarti Anthony Ferritto Michael R. Glass A. Gliozzo Salim Roukos Radu Florian Avirup Sil 11 14 0 11 Sep 2019
Span Selection Pre-training for Question Answering Michael R. Glass A. Gliozzo Rishav Chakravarti Anthony Ferritto Lin Pan G P Shrivatsa Bhargav Dinesh Garg Avirup Sil RALM 30 70 0 09 Sep 2019
Pretrained Language Models for Sequential Sentence Classification Arman Cohan Iz Beltagy Daniel King Bhavana Dalvi Daniel S. Weld 24 128 0 09 Sep 2019
KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning Bill Yuchen Lin Xinyue Chen Jamin Chen Xiang Ren 13 459 0 04 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 40 11,663 0 27 Aug 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 43 826 0 25 Aug 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 29 1,647 0 22 Aug 2019
Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models Zhiquan Ye Qian Chen Wen Wang Zhenhua Ling 17 68 0 19 Aug 2019
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training Gen Li Nan Duan Yuejian Fang Ming Gong Daxin Jiang Ming Zhou SSL VLM MLLM 57 895 0 16 Aug 2019
Reasoning Over Paragraph Effects in Situations Kevin Lin Oyvind Tafjord Peter Clark Matt Gardner 11 115 0 16 Aug 2019
On Identifiability in Transformers Gino Brunner Yang Liu Damian Pascual Oliver Richter Massimiliano Ciaramita Roger Wattenhofer ViT 22 186 0 12 Aug 2019
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks S. Rothe Shashi Narayan Aliaksei Severyn SILM 63 433 0 29 Jul 2019
BERTphone: Phonetically-Aware Encoder Representations for Utterance-Level Speaker and Language Recognition Shaoshi Ling Julian Salazar Yuzong Liu Katrin Kirchhoff SSL 16 27 0 30 Jun 2019
Taming Pretrained Transformers for Extreme Multi-label Text Classification Wei-Cheng Chang Hsiang-Fu Yu Kai Zhong Yiming Yang Inderjit Dhillon 17 20 0 07 May 2019
Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches Shane Storks Qiaozi Gao J. Chai 13 128 0 02 Apr 2019
Sentence transition matrix: An efficient approach that preserves sentence semantics Myeongjun Jang Pilsung Kang 11 2 0 16 Jan 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018