RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,303 papers shown

Title
Capturing Structural Locality in Non-parametric Language Models Frank F. Xu Junxian He Graham Neubig Vincent J. Hellendoorn 16 14 0 06 Oct 2021
8-bit Optimizers via Block-wise Quantization Tim Dettmers M. Lewis Sam Shleifer Luke Zettlemoyer MQ 17 268 0 06 Oct 2021
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier Linyang Li Demin Song Ruotian Ma Xipeng Qiu Xuanjing Huang 27 21 0 06 Oct 2021
BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models Kangjie Chen Yuxian Meng Xiaofei Sun Shangwei Guo Tianwei Zhang Jiwei Li Chun Fan SILM 23 105 0 06 Oct 2021
Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Coding Saurabhchand Bhati Jesús Villalba Piotr Żelasko Laureano Moro Velázquez Najim Dehak SSL 53 22 0 05 Oct 2021
Exploring Conditional Text Generation for Aspect-Based Sentiment Analysis Siva Uday Sampreeth Chebolu Franck Dernoncourt Nedim Lipka Thamar Solorio 31 7 0 05 Oct 2021
Co-training an Unsupervised Constituency Parser with Weak Supervision Nickil Maveli Shay B. Cohen SSL 41 3 0 05 Oct 2021
Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy Yi Zhou Danushka Bollegala 31 9 0 05 Oct 2021
Analyzing the Impact of COVID-19 on Economy from the Perspective of Users Reviews Fatemeh Salmani H. Vahdat-Nejad H. Hajiabadi 16 5 0 05 Oct 2021
A Survey On Neural Word Embeddings Erhan Sezerer Selma Tekir AI4TS 21 12 0 05 Oct 2021
Classification of hierarchical text using geometric deep learning: the case of clinical trials corpus Sohrab Ferdowsi Nikolay Borissov J. Knafou P. Amini Douglas Teodoro 16 7 0 04 Oct 2021
Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning Priya Basu Tiasa Singha Roy Rakshit Naidu Zumrut Muftuoglu 22 20 0 04 Oct 2021
Towards Theme Detection in Personal Finance Questions John X. Qiu Adam Faulkner Aysu Ezen-Can 11 2 0 04 Oct 2021
LEMON: Explainable Entity Matching Nils Barlaug FAtt AAML 12 9 0 01 Oct 2021
SlovakBERT: Slovak Masked Language Model Matúš Pikuliak Stefan Grivalsky Martin Konopka Miroslav Blšták Martin Tamajka Viktor Bachratý Marián Simko Pavol Balázik Michal Trnka Filip Uhlárik 27 25 0 30 Sep 2021
Analysing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets Changchang Zeng Shaobo Li 16 6 0 29 Sep 2021
Template-free Prompt Tuning for Few-shot NER Ruotian Ma Xin Zhou Tao Gui Y. Tan Linyang Li Qi Zhang Xuanjing Huang VLM 143 177 0 28 Sep 2021
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations Fangyu Liu Yunlong Jiao Jordan Massiah Emine Yilmaz Serhii Havrylov SSL 87 29 0 27 Sep 2021
Context-guided Triple Matching for Multiple Choice Question Answering Xun Yao Junlong Ma Xinrong Hu Junping Liu Jie Yang Wanqing Li 14 2 0 27 Sep 2021
MFAQ: a Multilingual FAQ Dataset Maxime De Bruyn Ehsan Lotfi Jeska Buhmann Walter Daelemans RALM 42 21 0 27 Sep 2021
Rumour Detection via Zero-shot Cross-lingual Transfer Learning Lin Tian Xiuzhen Zhang Jey Han Lau 36 13 0 27 Sep 2021
QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions Daniel Weiss Paul Roit Ayal Klein Ori Ernst Ido Dagan 20 18 0 26 Sep 2021
Parallel Refinements for Lexically Constrained Text Generation with BART Xingwei He 21 39 0 26 Sep 2021
DziriBERT: a Pre-trained Language Model for the Algerian Dialect Amine Abdaoui Mohamed Berrimi Mourad Oussalah A. Moussaoui 32 43 0 25 Sep 2021
Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features Bruce W. Lee Yoonna Jang J. Lee VLM 33 75 0 25 Sep 2021
Is the Number of Trainable Parameters All That Actually Matters? A. Chatelain Amine Djeghri Daniel Hesslow Julien Launay Iacopo Poli 43 7 0 24 Sep 2021
Dense Contrastive Visual-Linguistic Pretraining Lei Shi Kai Shuang Shijie Geng Peng Gao Zuohui Fu Gerard de Melo Yunpeng Chen Sen Su VLM SSL 52 10 0 24 Sep 2021
Automated Fact-Checking: A Survey Xia Zeng Amani S. Abumansour A. Zubiaga HILM 175 94 0 23 Sep 2021
Named Entity Recognition and Classification on Historical Documents: A Survey Maud Ehrmann Ahmed Hamdi Elvys Linhares Pontes Matteo Romanello A. Doucet 52 108 0 23 Sep 2021
WRENCH: A Comprehensive Benchmark for Weak Supervision Jieyu Zhang Yue Yu Yinghao Li Yujing Wang Yaming Yang Mao Yang Alexander Ratner 8 110 0 23 Sep 2021
Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing K. Kanakarajan Bhuvana Kundumani Malaikannan Sankarasubbu ALM MoE 11 5 0 22 Sep 2021
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization Xinnuo Xu Ondrej Dusek Shashi Narayan Verena Rieser Ioannis Konstas HILM 23 6 0 22 Sep 2021
Caption Enriched Samples for Improving Hateful Memes Detection Efrat Blaier Itzik Malkiel Lior Wolf VLM 51 21 0 22 Sep 2021
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering Fu Sun Feng-Lin Li Ruize Wang Qianglong Chen Xingyi Cheng Ji Zhang VLM KELM 22 4 0 22 Sep 2021
FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning Xu Wang Hainan Zhang Shuai Zhao Yanyan Zou Hongshen Chen Zhuoye Ding Bo Cheng Yanyan Lan AAML 11 7 0 22 Sep 2021
Digital Signal Processing Using Deep Neural Networks Brian Shevitski Y. Watkins Nicole Man Michael Girard AI4CE 13 4 0 21 Sep 2021
BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese Nguyen Luong Tran Duong Minh Le Dat Quoc Nguyen 19 51 0 20 Sep 2021
Commonsense Knowledge in Word Associations and ConceptNet Chunhua Liu Trevor Cohn Lea Frermann 14 7 0 20 Sep 2021
Conditional probing: measuring usable information beyond a baseline John Hewitt Kawin Ethayarajh Percy Liang Christopher D. Manning 31 55 0 19 Sep 2021
Towards Zero-Label Language Learning Zirui Wang Adams Wei Yu Orhan Firat Yuan Cao SyDa 180 102 0 19 Sep 2021
Knowledge-Enhanced Evidence Retrieval for Counterargument Generation Yohan Jo Haneul Yoo Jinyeong Bak Alice H. Oh Chris Reed Eduard H. Hovy RALM 38 12 0 19 Sep 2021
Text Detoxification using Large Pre-trained Neural Models David Dale Anton Voronov Daryna Dementieva V. Logacheva Olga Kozlova Nikita Semenov Alexander Panchenko 39 71 0 18 Sep 2021
Emily: Developing An Emotion-affective Open-Domain Chatbot with Knowledge Graph-based Persona Weixuan Wang Xiaoling Cai Chongxuan Huang Haoran Wang H. Lu Ximing Liu Wei Peng AI4MH 36 3 0 18 Sep 2021
Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes Hyunwoo J. Kim Byeongchang Kim Gunhee Kim 40 67 0 18 Sep 2021
Towards Zero and Few-shot Knowledge-seeking Turn Detection in Task-orientated Dialogue Systems Di Jin Shuyang Gao Seokhwan Kim Yang Liu Dilek Z. Hakkani-Tür 16 7 0 18 Sep 2021
Relating Neural Text Degeneration to Exposure Bias Ting-Rui Chiang Yun-Nung Chen 45 17 0 17 Sep 2021
Neural Unification for Logic Reasoning over Natural Language Gabriele Picco Hoang Thanh Lam M. Sbodio Vanessa Lopez Garcia NAI LRM 16 13 0 17 Sep 2021
Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers Jason Phang Haokun Liu Samuel R. Bowman 22 25 0 17 Sep 2021
Language Models as a Knowledge Source for Cognitive Agents R. Wray James R. Kirk John E. Laird 11 15 0 17 Sep 2021
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection Matthew Matero Nikita Soni Niranjan Balasubramanian H. A. Schwartz 21 21 0 16 Sep 2021