DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

38 / 138 papers shown

Title
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation KAI-QING Zhou Kai Zheng Connor Pryor Yilin Shen Hongxia Jin Lise Getoor X. Wang 18 107 0 30 Jan 2023
OPD@NL4Opt: An ensemble approach for the NER task of the optimization problem Kangxu Wang Ze Chen Jiewen Zheng 13 6 0 06 Jan 2023
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning Wenhao Wu Wei Li Xinyan Xiao Jiachen Liu Sujian Li Yajuan Lv HILM 18 4 0 20 Dec 2022
Rethinking Label Smoothing on Multi-hop Question Answering Zhangyue Yin Yuxin Wang Xiannian Hu Yiguang Wu Hang Yan Xinyu Zhang Zhao Cao Xuanjing Huang Xipeng Qiu 12 9 0 19 Dec 2022
Latent Diffusion for Language Generation Justin Lovelace Varsha Kishore Chao-gang Wan Eliot Shekhtman Kilian Q. Weinberger DiffM 24 71 0 19 Dec 2022
Understanding BLOOM: An empirical study on diverse NLP tasks Parag Dakle Sai Krishna Rallabandi Preethi Raghavan AI4CE 31 3 0 27 Nov 2022
A Report on the Euphemisms Detection Shared Task Patrick Lee Anna Feldman J. Peng 33 9 0 23 Nov 2022
A Universal Discriminator for Zero-Shot Generalization Haike Xu Zongyu Lin Jing Zhou Yanan Zheng Zhilin Yang AI4CE 13 14 0 15 Nov 2022
Measuring Reliability of Large Language Models through Semantic Consistency Harsh Raj Domenic Rosati S. Majumdar HILM 22 30 0 10 Nov 2022
Detecting Euphemisms with Literal Descriptions and Visual Imagery .Ilker Kesen Aykut Erdem Erkut Erdem Iacer Calixto 21 4 0 08 Nov 2022
Using Deep Mixture-of-Experts to Detect Word Meaning Shift for TempoWiC Ze Chen Kangxu Wang Zijian Cai Jiewen Zheng Jiarong He Max Gao Jason Zhang MoE 14 3 0 07 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 25 34 0 01 Nov 2022
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking Yaoyiran Li Fangyu Liu Ivan Vulić Anna Korhonen 34 10 0 30 Oct 2022
Visual Answer Localization with Cross-modal Mutual Knowledge Transfer Yixuan Weng Bin Li 16 6 0 26 Oct 2022
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 24 116 0 20 Oct 2022
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective Adaku Uchendu Thai Le Dongwon Lee DeLMO 19 40 0 19 Oct 2022
Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detection Elisa Sanchez-Bayona Rodrigo Agerri 10 10 0 19 Oct 2022
MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation Detection Cagri Toraman Oguzhan Ozcelik Furkan Şahinuç Fazli Can 25 12 0 11 Oct 2022
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding Haoming Jiang Tianyu Cao Zheng Li Cheng-hsin Luo Xianfeng Tang Qingyu Yin Danqing Zhang R. Goutam Bing Yin RALM 16 11 0 08 Oct 2022
PART: Pre-trained Authorship Representation Transformer Javier Huertas-Tato Álvaro Huertas-García Alejandro Martín 21 8 0 30 Sep 2022
Scope of Pre-trained Language Models for Detecting Conflicting Health Information Josepho D. Gatto Madhusudan Basak S. Preum 27 7 0 22 Sep 2022
Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios Mana Ashida Saku Sugawara 51 6 0 16 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
Generating Intermediate Steps for NLI with Next-Step Supervision Deepanway Ghosal Somak Aditya Monojit Choudhury LRM 35 1 0 31 Aug 2022
Predicting Query-Item Relationship using Adversarial Training and Robust Modeling Techniques Min Seok Kim 22 0 0 23 Aug 2022
A Cognitive Study on Semantic Similarity Analysis of Large Corpora: A Transformer-based Approach Praneeth Nemani Satyanarayana Vollala 11 0 0 24 Jul 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 30 46 0 14 Jul 2022
DIALOG-22 RuATD Generated Text Detection Narek Maloyan Bulat Nutfullin Eugene Ilyushin DeLMO 19 8 0 16 Jun 2022
Detecting Label Errors by using Pre-Trained Language Models Derek Chong Jenny Hong Christopher D. Manning NoLa 38 21 0 25 May 2022
Adversarial Training for High-Stakes Reliability Daniel M. Ziegler Seraphina Nix Lawrence Chan Tim Bauman Peter Schmidt-Nielsen ... Noa Nabeshima Benjamin Weinstein-Raun D. Haas Buck Shlegeris Nate Thomas AAML 25 59 0 03 May 2022
On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations Roy Schwartz Gabriel Stanovsky 22 24 0 27 Apr 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation Simiao Zuo Qingru Zhang Chen Liang Pengcheng He T. Zhao Weizhu Chen MoE 14 38 0 15 Apr 2022
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis Shamsuddeen Hassan Muhammad David Ifeoluwa Adelani Sebastian Ruder I. Ahmad Idris Abdulmumin ... Chris C. Emezue Saheed Abdul Anuoluwapo Aremu Alipio Jeorge P. Brazdil 27 95 0 20 Jan 2022
BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish Angelica Chen Nikita Nangia Vishakh Padmakumar Jason Phang Jana Thompson Phu Mon Htut Sam Bowman 212 367 0 15 Oct 2021
Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing K. Kanakarajan Bhuvana Kundumani Malaikannan Sankarasubbu ALM MoE 11 5 0 22 Sep 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 122 202 0 16 Feb 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018