Neural Machine Translation of Rare Words with Subword Units

31 August 2015

Papers citing "Neural Machine Translation of Rare Words with Subword Units"

50 / 3,808 papers shown

Title
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series Ge Zhang Scott Qu Jiaheng Liu Chenchen Zhang Chenghua Lin ... Zi-Kai Zhao Jiajun Zhang Wanli Ouyang Wenhao Huang Wenhu Chen ELM 43 44 0 29 May 2024
Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation Langlin Huang Yang Feng 39 1 0 29 May 2024
Language Generation with Strictly Proper Scoring Rules Chenze Shao Fandong Meng Yijin Liu Jie Zhou 70 5 0 29 May 2024
Contextual Position Encoding: Learning to Count What's Important O. Yu. Golovneva Tianlu Wang Jason Weston Sainbayar Sukhbaatar 53 28 0 29 May 2024
Federating Dynamic Models using Early-Exit Architectures for Automatic Speech Recognition on Heterogeneous Clients Mohamed Nabih Ali Alessio Brutti Daniele Falavigna 45 0 0 27 May 2024
Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP Vagrant Gautam Arjun Subramonian Anne Lauscher O. Keyes 40 6 0 27 May 2024
Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization Dixuan Wang Yanda Li Junyuan Jiang Zepeng Ding Ziqin Luo Guochao Jiang Jiaqing Liang Deqing Yang 34 11 0 27 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 88 3 0 24 May 2024
Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training Xianzhi Du Tom Gunter Xiang Kong Mark Lee Zirui Wang Aonan Zhang Nan Du Ruoming Pang MoE 35 0 0 23 May 2024
Super Tiny Language Models Dylan Hillier Leon Guertler Cheston Tan Palaash Agrawal Ruirui Chen Bobby Cheng 63 4 0 23 May 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations Ziqiao Ma Zekun Wang Joyce Chai 63 3 0 22 May 2024
Towards Retrieval-Augmented Architectures for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Alessandro Nicolosi Rita Cucchiara VLM 32 10 0 21 May 2024
What Have We Achieved on Non-autoregressive Translation? Yafu Li Huajian Zhang Jianhao Yan Yongjing Yin Yue Zhang 44 1 0 21 May 2024
A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges Huangjun Shen Liangying Shao Wenbo Li Zhibin Lan Zhanyu Liu Jinsong Su 44 2 0 21 May 2024
Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models Charles OÑeill Thang Bui 43 5 0 21 May 2024
Large Language Models Lack Understanding of Character Composition of Words Andrew Shin Kunitake Kaneko 34 8 0 18 May 2024
When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models Xianzheng Ma Yash Bhalgat Brandon Smart Shuai Chen Xinghui Li ... Matthias Nießner Ian D Reid Angel X. Chang Iro Laina V. Prisacariu LRM 37 13 0 16 May 2024
TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data Yihong Liu Chunlan Ma Haotian Ye Hinrich Schütze 36 4 0 16 May 2024
Chameleon: Mixed-Modal Early-Fusion Foundation Models Chameleon Team MLLM 62 266 0 16 May 2024
Matching domain experts by training from scratch on domain knowledge Xiaoliang Luo Guangzhi Sun Bradley C. Love LRM ALM 32 3 0 15 May 2024
Full Line Code Completion: Bringing AI to Desktop Anton Semenkin Vitaliy Bibaev Yaroslav Sokolov Kirill Krylov Alexey Kalina ... Mikhail Podvitskii Petr Surkov Yaroslav Golubev Nikita Povarov T. Bryksin 45 2 0 14 May 2024
Self-Distillation Improves DNA Sequence Inference Tong Yu Lei Cheng Ruslan Khalitov Erland Brandser Olsson Zhirong Yang SyDa 43 0 0 14 May 2024
Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Models Andrea Piergentili Beatrice Savoldi Matteo Negri L. Bentivogli 50 5 0 14 May 2024
Challenges and Opportunities in Text Generation Explainability Kenza Amara Rita Sevastjanova Mennatallah El-Assady SILM 48 2 0 14 May 2024
VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling Siyuan Li Zedong Wang Zicheng Liu Di Wu Cheng Tan Jiangbin Zheng Yufei Huang Stan Z. Li 45 7 0 13 May 2024
Zero-Shot Tokenizer Transfer Benjamin Minixhofer Edoardo Ponti Ivan Vulić VLM 49 9 0 13 May 2024
LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Language Cagri Toraman VLM 46 5 0 13 May 2024
Constructing a BPE Tokenization DFA Martin Berglund Willeke Martens Brink van der Merwe 22 2 0 13 May 2024
DualFocus: A Unified Framework for Integrating Positive and Negative Descriptors in Text-based Person Retrieval Yuchuan Deng Zhanpeng Hu Jiakun Han Chuang Deng Qijun Zhao 46 0 0 13 May 2024
Towards a More Inclusive AI: Progress and Perspectives in Large Language Model Training for the Sámi Language Ronny Paul Himanshu Buckchash Shantipriya Parida Dilip K. Prasad 35 2 0 09 May 2024
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models Sander Land Max Bartolo 49 21 0 08 May 2024
Revisiting character-level adversarial attacks Elias Abad Rocamora Yongtao Wu Fanghui Liu Grigorios G. Chrysos V. Cevher AAML 39 3 0 07 May 2024
Granite Code Models: A Family of Open Foundation Models for Code Intelligence Mayank Mishra Matt Stallone Gaoyuan Zhang Songlin Yang Aditya Prasad ... Amith Singhee Nirmit Desai David D. Cox Ruchir Puri Yikang Shen AI4TS 65 58 0 07 May 2024
DrugLLM: Open Large Language Model for Few-shot Molecule Generation Xianggen Liu Yan Guo Haoran Li Jin Liu Shudong Huang Bowen Ke Jiancheng Lv 34 6 0 07 May 2024
Context-Aware Machine Translation with Source Coreference Explanation Huy Hien Vu Hidetaka Kamigaito Taro Watanabe LRM 44 2 0 30 Apr 2024
Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition Solène Tarride Christopher Kermorvant 37 1 0 30 Apr 2024
Modeling Orthographic Variation in Occitan's Dialects Zachary Hopton Noemi Aepli 35 2 0 30 Apr 2024
Unknown Script: Impact of Script on Cross-Lingual Transfer Wondimagegnhue Tufa Ilia Markov Piek Vossen 45 0 0 29 Apr 2024
A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system Sunil Kumar Kopparapu Ashish Panda 31 0 0 29 Apr 2024
Can Perplexity Predict Fine-Tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali Nishant Luitel Nirajan Bekoju Anand Kumar Sah Subarna Shakya 58 1 0 28 Apr 2024
Scaffold-BPE: Enhancing Byte Pair Encoding with Simple and Effective Scaffold Token Removal Haoran Lian Yizhe Xiong Jianwei Niu Shasha Mo Zhenpeng Su Zijia Lin Peng Liu Hui Chen Guiguang Ding 44 1 0 27 Apr 2024
AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation Zhensu Sun Xiaoning Du Zhou Yang Li Li David Lo 32 10 0 25 Apr 2024
Act as a Honeytoken Generator! An Investigation into Honeytoken Generation with Large Language Models Daniel Reti Norman Becker Tillmann Angeli Anasuya Chattopadhyay Daniel Schneider Sebastian Vollmer Hans D. Schotten 40 5 0 24 Apr 2024
SpaceByte: Towards Deleting Tokenization from Large Language Modeling Kevin Slagle 37 3 0 22 Apr 2024
Intrusion Detection at Scale with the Assistance of a Command-line Language Model Jiongliang Lin Yiwen Guo Hao Chen 16 2 0 20 Apr 2024
HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Grounding Linhui Xiao Xiaoshan Yang Fang Peng Yaowei Wang Changsheng Xu ObjD 36 8 0 20 Apr 2024
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge Khuyagbaatar Batsuren Ekaterina Vylomova Verna Dankers Tsetsuukhei Delgerbaatar Omri Uzan Yuval Pinter Gábor Bella 42 10 0 20 Apr 2024
TartuNLP @ SIGTYP 2024 Shared Task: Adapting XLM-RoBERTa for Ancient and Historical Languages Aleksei Dorkin Kairit Sirts 38 1 0 19 Apr 2024
Large Language Models in Targeted Sentiment Analysis Nicolay Rusnachenko A. Golubev Natalia Loukachevitch LRM 32 3 0 18 Apr 2024
Gaining More Insight into Neural Semantic Parsing with Challenging Benchmarks Xiao Zhang Chunliu Wang Rik van Noord Johan Bos 36 3 0 12 Apr 2024