Morphological and Language-Agnostic Word Segmentation for NMT

14 June 2018

Papers citing "Morphological and Language-Agnostic Word Segmentation for NMT"

18 / 18 papers shown

Title
From Smør-re-brød to Subwords: Training LLMs on Danish, One Morpheme at a Time Mikkel Wildner Kildeberg Emil Allerslev Schledermann Nicolaj Larsen Rob van der Goot 111 0 0 02 Apr 2025
Why do language models perform worse for morphologically complex languages? Catherine Arnett Benjamin Bergen 144 22 0 21 Nov 2024
Unsupervised Morphological Tree Tokenizer Qingyang Zhu Xiang Hu Pengyu Ji Wei Wu Kewei Tu 129 0 0 21 Jun 2024
Lexically Grounded Subword Segmentation Jindřich Libovický Jindřich Helcl 170 7 0 19 Jun 2024
Low-resource neural machine translation with morphological modeling Antoine Nzeyimana 110 9 0 03 Apr 2024
mALBERT: Is a Compact Multilingual BERT Model Still Worth It? Christophe Servan Sahar Ghannay Sophie Rosset 88 1 0 27 Mar 2024
Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement Catherine Arnett Pamela D. Rivière Tyler A. Chang Sean Trott 123 4 0 20 Mar 2024
Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That! Niyati Bafna Philipp Koehn David Yarowsky 163 1 0 16 Mar 2024
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling Tomasz Limisiewicz Terra Blevins Hila Gonen Orevaoghene Ahia Luke Zettlemoyer 120 23 0 15 Mar 2024
MorphPiece : A Linguistic Tokenizer for Large Language Models Jeffrey Hsu 85 6 0 14 Jul 2023
Tokenization with Factorized Subword Encoding David Samuel Lilja Øvrelid 90 2 0 13 Jun 2023
The SIGMORPHON 2022 Shared Task on Morpheme Segmentation Khuyagbaatar Batsuren Gábor Bella Aryaman Arora Viktor Martinović Kyle Gorman ... Magda vSevvcíková Katevrina Pelegrinová Fausto Giunchiglia Robert Bamler Ekaterina Vylomova 82 44 0 15 Jun 2022
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP Sabrina J. Mielke Zaid Alyafeai Elizabeth Salesky Colin Raffel Manan Dey ... Arun Raja Chenglei Si Wilson Y. Lee Benoît Sagot Samson Tan 186 167 0 20 Dec 2021
Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey Danielle Saunders AI4CE 196 100 0 14 Apr 2021
Crowdsourced Phrase-Based Tokenization for Low-Resourced Neural Machine Translation: The Case of Fon Language Bonaventure F. P. Dossou Chris C. Emezue 93 5 0 14 Mar 2021
Neural Machine Translation: A Review and Survey Felix Stahlberg 3DV AI4TS MedIm 204 353 0 04 Dec 2019
Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed Thuong-Hai Pham Dominik Machácek Ondrej Bojar 82 11 0 24 Oct 2019
Finding the Answers with Definition Models Jack Parry 36 0 0 01 Sep 2018