An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference

16 February 2024

Papers citing "An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference"

4 / 4 papers shown

Title
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization Enes Özeren Yihong Liu Hinrich Schütze 28 0 0 21 Apr 2025
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most? HyoJung Han Akiko Eriguchi Haoran Xu Hieu T. Hoang Marine Carpuat Huda Khayrallah VLM 32 2 0 12 Oct 2024
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 106 165 0 24 Oct 2020