UNKs Everywhere: Adapting Multilingual Language Models to New Scripts

31 December 2020

Papers citing "UNKs Everywhere: Adapting Multilingual Language Models to New Scripts"

30 / 30 papers shown

Title
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization Enes Özeren Yihong Liu Hinrich Schütze 31 0 0 21 Apr 2025
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation Vera Neplenbroek Arianna Bisazza Raquel Fernández 100 0 0 17 Feb 2025
Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages Hoang Nguyen Khyati Mahajan Vikas Yadav Philip S. Yu Masoud Hashemi Rishabh Maheshwary Rishabh Maheshwary 45 0 0 04 Nov 2024
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment Yongxin Huang Kexin Wang Goran Glavavs Iryna Gurevych 44 0 0 20 Jul 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 30 19 0 13 Jun 2024
Unknown Script: Impact of Script on Cross-Lingual Transfer Wondimagegnhue Tufa Ilia Markov Piek Vossen 24 0 0 29 Apr 2024
Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect Jannis Vamvas Noëmi Aepli Rico Sennrich 32 0 0 25 Jan 2024
LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization Muhammad Farid Adilazuarda Samuel Cahyawijaya Alham Fikri Aji Genta Indra Winata Ayu Purwarianti 19 5 0 11 Jan 2024
Efficient Domain Adaptation of Sentence Embeddings Using Adapters Tim Schopf Dennis Schneider Florian Matthes 26 5 0 06 Jul 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 23 23 0 03 Jul 2023
Having Beer after Prayer? Measuring Cultural Bias in Large Language Models Tarek Naous Michael Joseph Ryan Alan Ritter Wei-ping Xu 24 84 0 23 May 2023
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages Ayyoob Imani Peiqin Lin Amir Hossein Kargaran Silvia Severini Masoud Jalili Sabet ... Chunlan Ma Helmut Schmid André F. T. Martins François Yvon Hinrich Schütze ALM LRM 29 95 0 20 May 2023
Language Model Tokenizers Introduce Unfairness Between Languages Aleksandar Petrov Emanuele La Malfa Philip H. S. Torr Adel Bibi 16 96 0 17 May 2023
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing Chen Cecilia Liu Jonas Pfeiffer Ivan Vulić Iryna Gurevych CLL 24 9 0 13 Jan 2023
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 27 81 0 19 Dec 2022
Extending the Subwording Model of Multilingual Pretrained Models for New Languages K. Imamura Eiichiro Sumita VLM 13 3 0 29 Nov 2022
hmBERT: Historical Multilingual Language Models for Named Entity Recognition Stefan Schweter Luisa März Katharina Schmid Erion cCano 25 18 0 31 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 35 138 0 12 May 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 31 148 0 15 Apr 2022
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis Shamsuddeen Hassan Muhammad David Ifeoluwa Adelani Sebastian Ruder I. Ahmad Idris Abdulmumin ... Chris C. Emezue Saheed Abdul Anuoluwapo Aremu Alipio Jeorge P. Brazdil 27 95 0 20 Jan 2022
Continual Learning in Multilingual NMT via Language-Specific Embeddings Alexandre Berard CLL 23 20 0 20 Oct 2021
xGQA: Cross-Lingual Visual Question Answering Jonas Pfeiffer Gregor Geigle Aishwarya Kamath Jan-Martin O. Steitz Stefan Roth Ivan Vulić Iryna Gurevych 19 56 0 13 Sep 2021
Subword Mapping and Anchoring across Languages Giorgos Vernikos Andrei Popescu-Belis 62 12 0 09 Sep 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 35 73 0 01 Jul 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 23 27 0 16 Jun 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 234 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavas 29 27 0 11 Dec 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 93 142 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 58 65 0 24 Oct 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 19 254 0 22 Oct 2020