RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,675 papers shown

Title
Prototypical Verbalizer for Prompt-based Few-shot Tuning Ganqu Cui Shengding Hu Ning Ding Longtao Huang Zhiyuan Liu VLM 15 96 0 18 Mar 2022
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models Aaron Mueller Robert Frank Tal Linzen Luheng Wang Sebastian Schuster AIMat 19 33 0 17 Mar 2022
EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training Yuxian Gu Jiaxin Wen Hao-Lun Sun Yi Song Pei Ke ... Zheng Zhang Jianzhu Yao Lei Liu Xiaoyan Zhu Minlie Huang 19 55 0 17 Mar 2022
Finding Structural Knowledge in Multimodal-BERT Victor Milewski Miryam de Lhoneux Marie-Francine Moens 19 9 0 17 Mar 2022
Automating Code Review Activities by Large-Scale Pre-training Zhiyu Li Shuai Lu Daya Guo Nan Duan Shailesh Jannu ... Deep Majumder Jared Green Alexey Svyatkovskiy Shengyu Fu Neel Sundaresan VLM 23 139 0 17 Mar 2022
UNIMO-2: End-to-End Unified Vision-Language Grounded Learning Wei Li Can Gao Guocheng Niu Xinyan Xiao Hao Liu Jiachen Liu Hua-Hong Wu Haifeng Wang MLLM 11 21 0 17 Mar 2022
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training Luyang Huang Guocheng Niu Jiachen Liu Xinyan Xiao Hua-Hong Wu VLM CoGe 14 7 0 17 Mar 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Learning Audio Representations with MLPs Mashrur M. Morshed Ahmad Omar Ahsan H. Mahmud Md. Kamrul Hasan 24 4 0 16 Mar 2022
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning Jiangjie Chen Rui Xu Ziquan Fu Wei Shi Zhongqiao Li Xinbo Zhang Changzhi Sun Lei Li Yanghua Xiao Hao Zhou ELM 23 35 0 16 Mar 2022
HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction Zalan Fabian Berk Tinaz Mahdi Soltanolkotabi 25 50 0 15 Mar 2022
Does Corpus Quality Really Matter for Low-Resource Languages? Mikel Artetxe Itziar Aldabe Rodrigo Agerri Olatz Perez-de-Viñaspre Aitor Soroa Etxabe 41 19 0 15 Mar 2022
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding Meng Li Tianbao Wang Haoyu Zhang Shengyu Zhang Zhou Zhao ... Wenming Tan Jin Wang Peng Wang Shi Pu Fei Wu 21 45 0 15 Mar 2022
SCD: Self-Contrastive Decorrelation for Sentence Embeddings T. Klein Moin Nabi SSL 30 25 0 15 Mar 2022
Do Language Models Plagiarize? Jooyoung Lee Thai Le Jinghui Chen Dongwon Lee 27 73 0 15 Mar 2022
Choose Your QA Model Wisely: A Systematic Study of Generative and Extractive Readers for Question Answering Man Luo Kazuma Hashimoto Semih Yavuz Zhiwei Liu Chitta Baral Yingbo Zhou 21 21 0 14 Mar 2022
CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection Jens Lemmens Jens Van Nooten Tim Kreutz Walter Daelemans 14 6 0 14 Mar 2022
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing Ilias Chalkidis Tommaso Pasini Shenmin Zhang Letizia Tomada Sebastian Felix Schwemer Anders Søgaard AILaw 32 54 0 14 Mar 2022
A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification Dairui Liu Derek Greene Ruihai Dong 25 10 0 14 Mar 2022
Disentangled Representation Learning for Text-Video Retrieval Qiang Wang Yanhao Zhang Yun Zheng Pan Pan Xiansheng Hua 45 76 0 14 Mar 2022
S $^2$ SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers Binyuan Hui Ruiying Geng Lihan Wang Bowen Qin Bowen Li Jian Sun Yongbin Li 23 55 0 14 Mar 2022
PERT: Pre-training BERT with Permuted Language Model Yiming Cui Ziqing Yang Ting Liu 25 37 0 14 Mar 2022
Efficient Language Modeling with Sparse all-MLP Ping Yu Mikel Artetxe Myle Ott Sam Shleifer Hongyu Gong Ves Stoyanov Xian Li MoE 15 11 0 14 Mar 2022
Can pre-trained Transformers be used in detecting complex sensitive sentences? -- A Monsanto case study Roelien C. Timmer David Liebowitz Surya Nepal S. Kanhere 14 8 0 14 Mar 2022
SciNLI: A Corpus for Natural Language Inference on Scientific Text Mobashir Sadat Cornelia Caragea AILaw 24 35 0 13 Mar 2022
Efficient Long-Range Attention Network for Image Super-resolution Xindong Zhang Huiyu Zeng Shi Guo Lei Zhang ViT 19 276 0 13 Mar 2022
Masked Autoencoders for Point Cloud Self-supervised Learning Yatian Pang Wenxiao Wang Francis E. H. Tay W. Liu Yonghong Tian Liuliang Yuan 3DPC ViT 33 453 0 13 Mar 2022
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Mathieu Ravaut Shafiq R. Joty Nancy F. Chen MoE 13 91 0 13 Mar 2022
Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice Andreas Grivas Nikolay Bogoychev Adam Lopez 11 9 0 12 Mar 2022
What Makes Reading Comprehension Questions Difficult? Saku Sugawara Nikita Nangia Alex Warstadt Sam Bowman ELM RALM 20 13 0 12 Mar 2022
CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment Lutfi Kerem Senel Timo Schick Hinrich Schütze ELM ALM 23 5 0 11 Mar 2022
LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval Canwen Xu Daya Guo Nan Duan Julian McAuley RALM VLM 16 46 0 11 Mar 2022
Achieving Reliable Human Assessment of Open-Domain Dialogue Systems Tianbo Ji Yvette Graham Gareth J. F. Jones Chenyang Lyu Qun Liu ALM 29 39 0 11 Mar 2022
BERTopic: Neural topic modeling with a class-based TF-IDF procedure M. Grootendorst 39 1,375 0 11 Mar 2022
A comparative study of non-deep learning, deep learning, and ensemble learning methods for sunspot number prediction Yuchen Dang Ziqi Chen Heng Li Hai Shu ELM BDL 19 26 0 11 Mar 2022
LineVD: Statement-level Vulnerability Detection using Graph Neural Networks David Hin Andrey Kan Huaming Chen M. Babar 26 158 0 10 Mar 2022
Speciesist Language and Nonhuman Animal Bias in English Masked Language Models Masashi Takeshita Rafal Rzepka K. Araki 24 6 0 10 Mar 2022
HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing Sonish Sivarajkumar Yanshan Wang VLM LM&MA 31 54 0 09 Mar 2022
Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang Daphna Keidar Andreas Opedal Zhijing Jin Mrinmaya Sachan 22 20 0 09 Mar 2022
Measuring the Mixing of Contextual Information in the Transformer Javier Ferrando Gerard I. Gállego Marta R. Costa-jussá 23 48 0 08 Mar 2022
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks Fatemehsadat Mireshghallah Kartik Goyal Archit Uniyal Taylor Berg-Kirkpatrick Reza Shokri MIALM 30 151 0 08 Mar 2022
UniXcoder: Unified Cross-Modal Pre-training for Code Representation Daya Guo Shuai Lu Nan Duan Yanlin Wang Ming Zhou Jian Yin 14 559 0 08 Mar 2022
A Unified Framework of Medical Information Annotation and Extraction for Chinese Clinical Text Enwei Zhu Qilin Sheng Huan Yang Jinpeng Li 19 8 0 08 Mar 2022
Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers Miguel A. Saavedra-Ruiz Sacha Morin Liam Paull MDE ViT 35 3 0 07 Mar 2022
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models Shengnan An Yifei Li Zeqi Lin Qian Liu Bei Chen Qiang Fu Weizhu Chen Nanning Zheng Jian-Guang Lou VLM AAML 34 39 0 07 Mar 2022
Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents Yicheng Zou Hongwei Liu Tao Gui Junzhe Wang Qi Zhang M. Tang Haixiang Li Dan Wang DRL 35 29 0 06 Mar 2022
Leveraging Pre-trained BERT for Audio Captioning Xubo Liu Xinhao Mei Qiushi Huang Jianyuan Sun Jinzheng Zhao Haohe Liu Mark D. Plumbley Volkan Kilicc Wenwu Wang 25 29 0 06 Mar 2022
Just Rank: Rethinking Evaluation with Word and Sentence Similarities Bin Wang C.-C. Jay Kuo Haizhou Li ELM 19 29 0 05 Mar 2022
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification Yucheng Zhou Tao Shen Xiubo Geng Guodong Long Daxin Jiang 19 57 0 04 Mar 2022
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation Zheng Lian Lang Chen Licai Sun B. Liu J. Tao 25 83 0 04 Mar 2022