Neural Machine Translation of Rare Words with Subword Units

31 August 2015

Papers citing "Neural Machine Translation of Rare Words with Subword Units"

50 / 3,808 papers shown

Title
Investigating Neural Machine Translation for Low-Resource Languages: Using Bavarian as a Case Study Wan-Hua Her Udo Kruschwitz 45 4 0 12 Apr 2024
The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments Anton Schäfer Shauli Ravfogel Thomas Hofmann Tiago Pimentel Imanol Schlag 68 3 0 11 Apr 2024
Curated Datasets and Neural Models for Machine Translation of Informal Registers between Mayan and Spanish Vernaculars Andrés Lou Juan Antonio Pérez-Ortiz Felipe Sánchez-Martínez Víctor M. Sánchez-Cartagena 29 1 0 11 Apr 2024
Analyzing the Performance of Large Language Models on Code Summarization Rajarshi Haldar J. Hockenmaier 46 18 0 10 Apr 2024
DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with Latent Space Jianxiang Xiang Zhenhua Liu Haodong Liu Yin Bai Jia Cheng Wenliang Chen DiffM 29 2 0 10 Apr 2024
On the Effect of (Near) Duplicate Subwords in Language Modelling Anton Schäfer Thomas Hofmann Imanol Schlag Tiago Pimentel 47 1 0 09 Apr 2024
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies Shengding Hu Yuge Tu Xu Han Chaoqun He Ganqu Cui ... Chaochao Jia Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun MoE 51 298 0 09 Apr 2024
The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge Yiwei Guo Chenrun Wang Yifan Yang Hankun Wang Ziyang Ma ... Hanzheng Li Shuai Fan Hui Zhang Xie Chen Kai Yu 46 1 0 09 Apr 2024
Interplay of Machine Translation, Diacritics, and Diacritization Wei-Rui Chen Ife Adebara Muhammad Abdul-Mageed 51 0 0 09 Apr 2024
Contextual Chart Generation for Cyber Deception David D. Nguyen David Liebowitz Surya Nepal S. Kanhere Sharif Abuadbba 49 0 0 07 Apr 2024
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation Junhong Wu Yuchen Liu Chengqing Zong CLL 44 1 0 07 Apr 2024
Training LLMs over Neurally Compressed Text Brian Lester Jaehoon Lee A. Alemi Jeffrey Pennington Adam Roberts Jascha Narain Sohl-Dickstein Noah Constant 45 6 0 04 Apr 2024
Sailor: Open Language Models for South-East Asia Longxu Dou Qian Liu Guangtao Zeng Jia Guo Jiahui Zhou Wei Lu Min Lin LRM 40 9 0 04 Apr 2024
Revisiting subword tokenization: A case study on affixal negation in large language models Thinh Hung Truong Yulia Otmakhova Karin Verspoor Trevor Cohn Timothy Baldwin 47 2 0 03 Apr 2024
Low-resource neural machine translation with morphological modeling Antoine Nzeyimana 39 4 0 03 Apr 2024
Emergent Abilities in Reduced-Scale Generative Language Models Sherin Muckatira Vijeta Deshpande Vladislav Lialin Anna Rumshisky ReLM ELM LRM 41 4 0 02 Apr 2024
HyperCLOVA X Technical Report Kang Min Yoo Jaegeun Han Sookyo In Heewon Jeon Jisu Jeong ... Hyunkyung Noh Se-Eun Choi Sang-Woo Lee Jung Hwa Lim Nako Sung VLM 44 8 0 02 Apr 2024
Transfer Learning from Whisper for Microscopic Intelligibility Prediction Paul Best Santiago Cuervo R. Marxer 41 2 0 02 Apr 2024
Forklift: An Extensible Neural Lifter Jordi Armengol-Estapé Rodrigo C. O. Rocha Jackson Woodruff Pasquale Minervini Michael F. P. O'Boyle 35 0 0 01 Apr 2024
Bailong: Bilingual Transfer Learning based on QLoRA and Zip-tie Embedding Lung-Chuan Chen Zong-Ru Li ALM 42 0 0 01 Apr 2024
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning Eli Schwartz Leshem Choshen J. Shtok Sivan Doveh Leonid Karlinsky Assaf Arbelle 39 13 0 30 Mar 2024
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation Marco Cognetta Tatsuya Hiraoka Naoaki Okazaki Rico Sennrich Yuval Pinter 39 2 0 30 Mar 2024
A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation Francois Meyer Jan Buys 39 1 0 29 Mar 2024
Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets Shadi Manafi Nikhil Krishnaswamy AAML 48 0 0 29 Mar 2024
Jamba: A Hybrid Transformer-Mamba Language Model Opher Lieber Barak Lenz Hofit Bata Gal Cohen Jhonathan Osin ... Nir Ratner N. Rozen Erez Shwartz Mor Zusman Y. Shoham 36 208 0 28 Mar 2024
Homogeneous Tokenizer Matters: Homogeneous Visual Tokenizer for Remote Sensing Image Understanding Run Shao Zhaoyang Zhang Chao Tao Yunsheng Zhang Chengli Peng Haifeng Li VLM 48 5 0 27 Mar 2024
Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction Inhwan Bae Junoh Lee Hae-Gon Jeon 38 15 0 27 Mar 2024
BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text Elliot Bolton Abhinav Venigalla Michihiro Yasunaga David Leo Wright Hall Betty Xiong ... R. Daneshjou Jonathan Frankle Percy Liang Michael Carbin Christopher D. Manning LM&MA MedIm 32 52 0 27 Mar 2024
mALBERT: Is a Compact Multilingual BERT Model Still Worth It? Christophe Servan Sahar Ghannay Sophie Rosset 44 0 0 27 Mar 2024
Leveraging Large Language Models for Fuzzy String Matching in Political Science Yu Wang 31 0 0 27 Mar 2024
Juru: Legal Brazilian Large Language Model from Reputable Sources Roseval Malaquias Junior Ramon Pires R. Romero R. Nogueira ELM AILaw 34 0 0 26 Mar 2024
Provably Secure Disambiguating Neural Linguistic Steganography Yuang Qi Kejiang Chen Kai Zeng Weiming Zhang Neng H. Yu 26 2 0 26 Mar 2024
The Role of $n$ -gram Smoothing in the Age of Neural Networks Luca Malagutti Andrius Buinovskij Anej Svete Clara Meister Afra Amini Ryan Cotterell 43 6 0 25 Mar 2024
Cross-lingual Contextualized Phrase Retrieval Huayang Li Deng Cai Zhi Qu Qu Cui Hidetaka Kamigaito Lemao Liu Taro Watanabe 34 0 0 25 Mar 2024
Understanding Emergent Abilities of Language Models from the Loss Perspective Zhengxiao Du Aohan Zeng Yuxiao Dong Jie Tang UQCV LRM 73 47 0 23 Mar 2024
KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation Xindi Luo Zequn Sun Jing-xin Zhao Zhe Zhao Wei Hu KELM 24 4 0 22 Mar 2024
More than Just Statistical Recurrence: Human and Machine Unsupervised Learning of Māori Word Segmentation across Morphological Processes A. Varatharaj Simon Todd 22 0 0 21 Mar 2024
A Multimodal Approach to Device-Directed Speech Detection with Large Language Models Dominik Wagner Alexander W. Churchill Siddharth Sigtia Panayiotis Georgiou Matt Mirsamadi Aarshee Mishra Erik Marchi 49 6 0 21 Mar 2024
Reverse Training to Nurse the Reversal Curse O. Yu. Golovneva Zeyuan Allen-Zhu Jason Weston Sainbayar Sukhbaatar 48 33 0 20 Mar 2024
Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement Catherine Arnett Pamela D. Rivière Tyler A. Chang Sean Trott 26 2 0 20 Mar 2024
Do Not Worry if You Do Not Have Data: Building Pretrained Language Models Using Translationese Meet Doshi Raj Dabre Pushpak Bhattacharyya SyDa 39 2 0 20 Mar 2024
Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models Zhixue Zhao Nikolaos Aletras 39 3 0 19 Mar 2024
CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification Korbinian Randl John Pavlopoulos Aron Henriksson Tony Lindgren 44 3 0 18 Mar 2024
Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models M. Alrefaie Nour Eldin Morsy Nada Samir 27 6 0 17 Mar 2024
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling Tomasz Limisiewicz Terra Blevins Hila Gonen Orevaoghene Ahia Luke Zettlemoyer 32 13 0 15 Mar 2024
Using Contextual Information for Sentence-level Morpheme Segmentation Prabin Bhandari Abhishek Paudel 21 1 0 15 Mar 2024
Simple and Scalable Strategies to Continually Pre-train Large Language Models Adam Ibrahim Benjamin Thérien Kshitij Gupta Mats L. Richter Quentin Anthony Timothée Lesort Eugene Belilovsky Irina Rish KELM CLL 49 54 0 13 Mar 2024
Masked Generative Story Transformer with Character Guidance and Caption Augmentation Christos Papadimitriou Giorgos Filandrianos Maria Lymperaiou Giorgos Stamou DiffM 102 1 0 13 Mar 2024
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension Lei Zhu Fangyun Wei Yanye Lu MLLM VLM 54 18 0 12 Mar 2024
Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generation Francois Meyer Jan Buys 31 2 0 12 Mar 2024