From English To Foreign Languages: Transferring Pre-trained Language Models

18 February 2020

Papers citing "From English To Foreign Languages: Transferring Pre-trained Language Models"

31 / 31 papers shown

Title
Bielik v3 Small: Technical Report Krzysztof Ociepa Łukasz Flis Remigiusz Kinas Krzysztof Wróbel Adrian Gwoździej 27 0 0 05 May 2025
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization Enes Özeren Yihong Liu Hinrich Schütze 31 0 0 21 Apr 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 93 0 0 25 Mar 2025
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most? HyoJung Han Akiko Eriguchi Haoran Xu Hieu T. Hoang Marine Carpuat Huda Khayrallah VLM 32 2 0 12 Oct 2024
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks Alex Cloud Jacob Goldman-Wetzler Evžen Wybitul Joseph Miller Alexander Matt Turner 21 2 0 06 Oct 2024
An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models Nandini Mundra Aditya Nanda Kishore Raj Dabre Ratish Puduppully Anoop Kunchukuttan Mitesh Khapra 30 3 0 08 Jul 2024
Zero-Shot Tokenizer Transfer Benjamin Minixhofer E. Ponti Ivan Vulić VLM 44 9 0 13 May 2024
Bailong: Bilingual Transfer Learning based on QLoRA and Zip-tie Embedding Lung-Chuan Chen Zong-Ru Li ALM 19 0 0 01 Apr 2024
Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models M. Alrefaie Nour Eldin Morsy Nada Samir 19 6 0 17 Mar 2024
Transferring BERT Capabilities from High-Resource to Low-Resource Languages Using Vocabulary Matching Piotr Rybak 22 1 0 22 Feb 2024
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference Atsuki Yamaguchi Aline Villavicencio Nikolaos Aletras 19 7 0 16 Feb 2024
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining Yihong Liu Peiqin Lin Mingyang Wang Hinrich Schütze 19 21 0 15 Nov 2023
Extrapolating Large Language Models to Non-English by Aligning Languages Wenhao Zhu Yunzhe Lv Qingxiu Dong Fei Yuan Jingjing Xu Shujian Huang Lingpeng Kong Jiajun Chen Lei Li 23 64 0 09 Aug 2023
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer Alan Ansell E. Ponti Anna Korhonen Ivan Vulić 17 4 0 02 Jun 2023
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training Kelly Marchisio Patrick Lewis Yihong Chen Mikel Artetxe 22 16 0 20 Dec 2022
GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost Qingcheng Zeng Lucas Garay Peilin Zhou Dading Chong Yining Hua Jiageng Wu Yi-Cheng Pan Han Zhou Rob Voigt Jie Yang VLM 19 22 0 13 Nov 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 35 138 0 12 May 2022
Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models Terra Blevins Luke Zettlemoyer 24 85 0 17 Apr 2022
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies Zhengxuan Wu Alex Tamkin Isabel Papadimitriou 21 9 0 24 Feb 2022
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models Benjamin Minixhofer Fabian Paischer Navid Rekabsaz 22 72 0 13 Dec 2021
On the Universality of Deep Contextual Language Models Shaily Bhatt Poonam Goyal Sandipan Dandapat Monojit Choudhury Sunayana Sitaram ELM 15 5 0 15 Sep 2021
Subword Mapping and Anchoring across Languages Giorgos Vernikos Andrei Popescu-Belis 62 12 0 09 Sep 2021
Cross-lingual Transferring of Pre-trained Contextualized Language Models Zuchao Li Kevin Parnow Hai Zhao Zhuosheng Zhang Rui Wang Masao Utiyama Eiichiro Sumita 13 8 0 27 Jul 2021
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models Zhengxuan Wu Nelson F. Liu Christopher Potts 8 3 0 17 Apr 2021
Graph Convolutional Network for Swahili News Classification Alexandros Kastanos Tyler Martin GNN 37 3 0 16 Mar 2021
What makes multilingual BERT multilingual? Chi-Liang Liu Tsung-Yuan Hsu Yung-Sung Chuang Hung-yi Lee 22 6 0 20 Oct 2020
A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT Chi-Liang Liu Tsung-Yuan Hsu Yung-Sung Chuang Hung-yi Lee 16 14 0 20 Apr 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 25 1,450 0 27 Feb 2020
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 15 770 0 25 Oct 2019
Word Translation Without Parallel Data Alexis Conneau Guillaume Lample MarcÁurelio Ranzato Ludovic Denoyer Hervé Jégou 165 1,634 0 11 Oct 2017
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism Orhan Firat Kyunghyun Cho Yoshua Bengio LRM AIMat 206 622 0 06 Jan 2016