DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 138 papers shown

Title
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding Ahmad Idrissi-Yaghir Amin Dada Henning Schafer Kamyar Arzideh Giulia Baldini ... Peter A. Horn Christin Seifert F. Nensa Jens Kleesiek Christoph M. Friedrich AI4MH 29 2 0 08 Apr 2024
AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets Pietro Lesci Andreas Vlachos 31 2 0 08 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 38 6 0 05 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 68 0 03 Apr 2024
Jetsons at FinNLP 2024: Towards Understanding the ESG Impact of a News Article using Transformer-based Models Parag Dakle Alolika Gon Sihan Zha Liang Wang SaiKrishna Rallabandi Preethi Raghavan 19 1 0 30 Mar 2024
RealKIE: Five Novel Datasets for Enterprise Key Information Extraction Benjamin Townsend Madison May Christopher Wells SyDa 33 0 0 29 Mar 2024
ELLEN: Extremely Lightly Supervised Learning For Efficient Named Entity Recognition Haris Riaz Razvan-Gabriel Dumitru Mihai Surdeanu MU 33 0 0 26 Mar 2024
Uncovering Latent Human Wellbeing in Language Model Embeddings Pedro Freire ChengCheng Tan Adam Gleave Dan Hendrycks Scott Emmons 30 1 0 19 Feb 2024
Can We Verify Step by Step for Incorrect Answer Detection? Xin Xu Shizhe Diao Can Yang Yang Wang LRM 122 13 0 16 Feb 2024
AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes Juhwan Choi Kyohoon Jin Junho Lee Sangmin Song Youngbin Kim 18 1 0 08 Feb 2024
Learning Shortcuts: On the Misleading Promise of NLU in Language Models Geetanjali Bihani Julia Taylor Rayz 27 3 0 17 Jan 2024
Authorship Obfuscation in Multilingual Machine-Generated Text Detection Dominik Macko Robert Moro Adaku Uchendu Ivan Srba Jason Samuel Lucas Michiharu Yamashita Nafis Irtiza Tripto Dongwon Lee Jakub Simko M. Bieliková DeLMO 32 17 0 15 Jan 2024
A Multi-Task, Multi-Modal Approach for Predicting Categorical and Dimensional Emotions Alex-Răzvan Ispas Théo Deschamps-Berger Laurence Devillers 40 1 0 31 Dec 2023
Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks Bo Li Wei Ye Quan-ding Wang Wen Zhao Shikun Zhang VLM 30 1 0 14 Dec 2023
Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals Yanai Elazar Bhargavi Paranjape Hao Peng Sarah Wiegreffe Khyathi Raghavi Vivek Srikumar Sameer Singh Noah A. Smith AAML OOD 21 0 0 16 Nov 2023
MELA: Multilingual Evaluation of Linguistic Acceptability Ziyin Zhang Yikang Liu Wei Huang Junyu Mao Rui Wang Hai Hu 22 3 0 15 Nov 2023
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models Keming Lu Hongyi Yuan Runji Lin Junyang Lin Zheng Yuan Chang Zhou Jingren Zhou MoE LRM 40 52 0 15 Nov 2023
Mirror: A Universal Framework for Various Information Extraction Tasks Tong Zhu Junfei Ren Zijian Yu Mengsong Wu Guoliang Zhang Xiaoye Qu Wenliang Chen Zhefeng Wang Baoxing Huai Min Zhang 29 14 0 09 Nov 2023
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning Chong Li Shaonan Wang Yunhao Zhang Jiajun Zhang Chengqing Zong 25 4 0 16 Oct 2023
Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text Chanho Park Chengsong Lu Mingjie Chen Thomas Hain 18 3 0 12 Oct 2023
Advancing Transformer's Capabilities in Commonsense Reasoning Yu Zhou Yunqiu Han Hanyu Zhou Yulun Wu VLM LRM ReLM 14 0 0 10 Oct 2023
FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics Yupei Du Albert Gatt Dong Nguyen 19 1 0 10 Oct 2023
IDTraffickers: An Authorship Attribution Dataset to link and connect Potential Human-Trafficking Operations on Text Escort Advertisements V. Saxena Benjamin Bashpole Gijs Van Dijck Gerasimos Spanakis 37 2 0 09 Oct 2023
SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References Matteo Gabburo Siddhant Garg Rik Koncel-Kedziorski Alessandro Moschitti 25 1 0 21 Sep 2023
Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains A. Sarvazyan José Ángel González Marc Franco-Salvador Francisco Rangel Berta Chulvi Paolo Rosso DeLMO 23 60 0 20 Sep 2023
From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models Masahiro Suzuki Masanori Hirano Hiroki Sakaji 39 6 0 07 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 36 32 0 23 Aug 2023
A Survey on Fairness in Large Language Models Yingji Li Mengnan Du Rui Song Xin Wang Ying Wang ALM 37 59 0 20 Aug 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 102 14 0 17 Aug 2023
Chain of Thought Prompting Elicits Knowledge Augmentation Di Wu Jing Zhang Xinmei Huang LRM 26 31 0 04 Jul 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 28 23 0 03 Jul 2023
Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory Masashi Takeshita Rafal Rzepka K. Araki 13 8 0 20 Jun 2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li Yifan Yu Qingru Zhang Chen Liang Pengcheng He Weizhu Chen Tuo Zhao 30 65 0 20 Jun 2023
LCT-1 at SemEval-2023 Task 10: Pre-training and Multi-task Learning for Sexism Detection and Classification K. Chernyshev E. Garanina Duygu Bayram Qiankun Zheng Lukas Edman 9 0 0 08 Jun 2023
A Unified One-Step Solution for Aspect Sentiment Quad Prediction Junxian Zhou Haiqin Yang Yuxuan He Hao Mou Junbo Yang 18 11 0 07 Jun 2023
CL-UZH at SemEval-2023 Task 10: Sexism Detection through Incremental Fine-Tuning and Multi-Task Learning with Label Descriptions Janis Goldzycher 11 1 0 06 Jun 2023
Reward Collapse in Aligning Large Language Models Ziang Song Tianle Cai Jason D. Lee Weijie J. Su ALM 21 22 0 28 May 2023
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering Yung-Sung Chuang Wei Fang Shang-Wen Li Wen-tau Yih James R. Glass LRM 22 12 0 26 May 2023
Learning Answer Generation using Supervision from Automatic Question Answering Evaluators Matteo Gabburo Siddhant Garg Rik Koncel-Kedziorski Alessandro Moschitti 21 6 0 24 May 2023
Continually Improving Extractive QA via Human Feedback Ge Gao Hung-Ting Chen Yoav Artzi Eunsol Choi 24 12 0 21 May 2023
Small Models are Valuable Plug-ins for Large Language Models Canwen Xu Yichong Xu Shuohang Wang Yang Liu Chenguang Zhu Julian McAuley LLMAG 36 44 0 15 May 2023
Using Language Models to Detect Alarming Student Responses Christopher M. Ormerod Milan Patel Harry Wang 32 0 0 12 May 2023
WikiSQE: A Large-Scale Dataset for Sentence Quality Estimation in Wikipedia Kenichiro Ando Satoshi Sekine Mamoru Komachi 15 2 0 10 May 2023
NLP-LTU at SemEval-2023 Task 10: The Impact of Data Augmentation and Semi-Supervised Learning Techniques on Text Classification Performance on an Imbalanced Dataset Sana Al-Azzawi Gyorgy Kovács Filip Nilsson Tosin P. Adewumi Marcus Liwicki 7 6 0 25 Apr 2023
Adapting Pretrained Language Models for Solving Tabular Prediction Problems in the Electronic Health Record C. McMaster D. Liew Douglas E. V. Pires 24 4 0 27 Mar 2023
SemEval-2023 Task 10: Explainable Detection of Online Sexism Hannah Rose Kirk Wenjie Yin Bertie Vidgen Paul Röttger 10 117 0 07 Mar 2023
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers Jon Saad-Falcon Omar Khattab Keshav Santhanam Radu Florian M. Franz Salim Roukos Avirup Sil Md Arafat Sultan Christopher Potts 13 41 0 01 Mar 2023
Automated Extraction of Fine-Grained Standardized Product Information from Unstructured Multilingual Web Data Alexander Flick Sebastian Jäger Ivana Trajanovska F. Biessmann 11 0 0 23 Feb 2023
ChatGPT: Jack of all trades, master of none Jan Kocoñ Igor Cichecki Oliwier Kaszyca Mateusz Kochanek Dominika Szydło ... Maciej Piasecki Lukasz Radliñski Konrad Wojtasik Stanislaw Wo'zniak Przemyslaw Kazienko AI4MH 15 526 0 21 Feb 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 16 24 0 19 Feb 2023