Mini-Model Adaptation: Efficiently Extending Pretrained Models to New
Languages via Aligned Shallow Training

Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training

20 December 2022

Kelly Marchisio

Patrick Lewis

Papers citing "Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training"

7 / 7 papers shown

Title
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization Enes Özeren Yihong Liu Hinrich Schütze 28 0 0 21 Apr 2025
FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models Xiaochen Wang Jiaqi Wang Houping Xiao J. Chen Fenglong Ma MedIm 61 7 0 17 Aug 2024
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 17 23 0 03 Jul 2023
An Efficient Multilingual Language Model Compression through Vocabulary Trimming Asahi Ushio Yi Zhou Jose Camacho-Collados 35 7 0 24 May 2023
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 106 165 0 24 Oct 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 239 489 0 16 Oct 2019