RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,487 papers shown

Title
Multi-document Summarization via Deep Learning Techniques: A Survey Congbo Ma W. Zhang Mingyu Guo Hu Wang Quan Z. Sheng 13 125 0 10 Nov 2020
Bangla Text Classification using Transformers Tanvirul Alam A. Khan Firoj Alam 15 34 0 09 Nov 2020
Towards Domain-Agnostic Contrastive Learning Vikas Verma Minh-Thang Luong Kenji Kawaguchi Hieu H. Pham Quoc V. Le SSL 15 115 0 09 Nov 2020
Positional Artefacts Propagate Through Masked Language Model Embeddings Ziyang Luo Artur Kulmizev Xiaoxi Mao 22 41 0 09 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 48 669 0 06 Nov 2020
Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining Grigorii Guz Patrick Huber Giuseppe Carenini 27 11 0 06 Nov 2020
EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answering Momchil Hardalov Todor Mihaylov Dimitrina Zlatkova Yoan Dinkov Ivan Koychev Preslav Nakov AI4Ed ELM 31 50 0 05 Nov 2020
Detecting Hallucinated Content in Conditional Neural Sequence Generation Chunting Zhou Graham Neubig Jiatao Gu Mona T. Diab P. Guzmán Luke Zettlemoyer Marjan Ghazvininejad HILM 36 194 0 05 Nov 2020
Indic-Transformers: An Analysis of Transformer Language Models for Indian Languages Kushal Kumar Jain Adwait Deshpande Kumar Shridhar F. Laumann Ayushman Dash 43 51 0 04 Nov 2020
The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks Brihi Joshi Neil Shah Francesco Barbieri Leonardo Neves 31 5 0 02 Nov 2020
Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets Seid Muhie Yimam A. Ayele Gopalakrishnan Venkatesh Ibrahim Gashaw Christian Biemann 20 27 0 02 Nov 2020
On the Sentence Embeddings from Pre-trained Language Models Bohan Li Hao Zhou Junxian He Mingxuan Wang Yiming Yang Lei Li 6 213 0 02 Nov 2020
Emergent Communication Pretraining for Few-Shot Machine Translation Yaoyiran Li E. Ponti Ivan Vulić Anna Korhonen 23 19 0 02 Nov 2020
ABNIRML: Analyzing the Behavior of Neural IR Models Sean MacAvaney Sergey Feldman Nazli Goharian Doug Downey Arman Cohan 15 49 0 02 Nov 2020
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias Marion Bartl Malvina Nissim Albert Gatt 14 122 0 27 Oct 2020
Probing Task-Oriented Dialogue Representation from Language Models Chien-Sheng Wu Caiming Xiong 8 20 0 26 Oct 2020
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference Jianguo Zhang Kazuma Hashimoto Wenhao Liu Chien-Sheng Wu Yao Wan Philip S. Yu R. Socher Caiming Xiong 14 92 0 25 Oct 2020
Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation Mrigank Raman Aaron Chan Siddhant Agarwal Peifeng Wang Hansen Wang Sungchul Kim Ryan Rossi Handong Zhao Nedim Lipka Xiang Ren 30 19 0 24 Oct 2020
Large Scale Legal Text Classification Using Transformer Models Zein Shaheen G. Wohlgenannt Erwin Filtz AILaw 21 67 0 24 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 93 142 0 24 Oct 2020
Structure-Grounded Pretraining for Text-to-SQL Xiang Deng Ahmed Hassan Awadallah Christopher Meek Oleksandr Polozov Huan Sun Matthew Richardson LMTD 13 150 0 24 Oct 2020
Adding Chit-Chat to Enhance Task-Oriented Dialogues Kai Sun Seungwhan Moon Paul A. Crook Stephen Roller Becka Silvert Bing-Quan Liu Zhiguang Wang Honglei Liu Eunjoon Cho Claire Cardie 67 66 0 24 Oct 2020
Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality Gustavo Aguilar Bryan McCann Tong Niu Nazneen Rajani N. Keskar Thamar Solorio 42 12 0 24 Oct 2020
ANLIzing the Adversarial Natural Language Inference Dataset Adina Williams Tristan Thrush Douwe Kiela AAML 166 45 0 24 Oct 2020
A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing Chunchuan Lyu Shay B. Cohen Ivan Titov 24 11 0 23 Oct 2020
Posterior Differential Regularization with f-divergence for Improving Model Robustness Hao Cheng Xiaodong Liu L. Pereira Yaoliang Yu Jianfeng Gao 240 31 0 23 Oct 2020
Concealed Data Poisoning Attacks on NLP Models Eric Wallace Tony Zhao Shi Feng Sameer Singh SILM 11 18 0 23 Oct 2020
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 51 0 23 Oct 2020
Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classification Linyi Yang Eoin M. Kenny T. L. J. Ng Yi Yang Barry Smyth Ruihai Dong 13 70 0 23 Oct 2020
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures N. Moosavi M. Boer Prasetya Ajie Utama Iryna Gurevych 19 13 0 23 Oct 2020
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification Francesco Barbieri Jose Camacho-Collados Leonardo Neves Luis Espinosa-Anke VLM 11 698 0 23 Oct 2020
Deep Learning Framework for Measuring the Digital Strategy of Companies from Earnings Calls A. Al-Ali R. Phaal D. Sull 13 3 0 23 Oct 2020
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing Ohad Rubin Jonathan Berant 133 150 0 23 Oct 2020
Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond Xin Li Lidong Bing Wenxuan Zhang Zheng Li Wai Lam 42 25 0 23 Oct 2020
Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets Gaurish Thakkar Marcis Pinnis 55 9 0 23 Oct 2020
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding Dongling Xiao Yukun Li Han Zhang Yu Sun Hao Tian Hua-Hong Wu Haifeng Wang 19 38 0 23 Oct 2020
Language Models are Open Knowledge Graphs Chenguang Wang Xiao Liu D. Song SSL KELM 24 135 0 22 Oct 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 46 254 0 22 Oct 2020
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval Akari Asai Eunsol Choi RALM 37 51 0 22 Oct 2020
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Lingkai Kong Haoming Jiang Yuchen Zhuang Jie Lyu T. Zhao Chao Zhang OODD 19 26 0 22 Oct 2020
Knowledge Distillation for BERT Unsupervised Domain Adaptation Minho Ryu K. Lee 18 35 0 22 Oct 2020
Mixed-Precision Embedding Using a Cache J. Yang Jianyu Huang Jongsoo Park P. T. P. Tang Andrew Tulloch 6 36 0 21 Oct 2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou Kevin Huang Tengyu Ma Jing Huang 16 273 0 21 Oct 2020
Knowledge Distillation for Improved Accuracy in Spoken Question Answering Chenyu You Nuo Chen Yuexian Zou 4 49 0 21 Oct 2020
Complaint Identification in Social Media with Transformer Networks Mali Jin Nikolaos Aletras 10 16 0 21 Oct 2020
German's Next Language Model Branden Chan Stefan Schweter Timo Möller 22 263 0 21 Oct 2020
"Are you home alone?" "Yes" Disclosing Security and Privacy Vulnerabilities in Alexa Skills Dan Su Jiqiang Liu Sencun Zhu Xiaoyang Wang Wei Wang ELM 31 17 0 21 Oct 2020
Transition-based Parsing with Stack-Transformers Ramón Fernández Astudillo Miguel Ballesteros Tahira Naseem Austin Blodgett Radu Florian 46 71 0 20 Oct 2020
An Empirical Investigation of Contextualized Number Prediction Daniel M. Spokoyny Taylor Berg-Kirkpatrick AI4TS 14 34 0 20 Oct 2020
AutoMeTS: The Autocomplete for Medical Text Simplification Hoang Van David Kauchak Gondy Leroy 42 30 0 20 Oct 2020