MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization

11 July 2024

Papers citing "MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization"

6 / 6 papers shown

Title
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 1 0 17 Mar 2025
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 30 2 0 28 Oct 2024
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 119 165 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 58 65 0 24 Oct 2020
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 75 5,262 0 03 Nov 2016