Neural Machine Translation of Rare Words with Subword Units

31 August 2015

Papers citing "Neural Machine Translation of Rare Words with Subword Units"

50 / 3,808 papers shown

Title
How to Understand Named Entities: Using Common Sense for News Captioning Ning Xu Yanhui Wang Tingting Zhang Hongshuo Tian Mohan Kankanhalli An-An Liu 40 0 0 11 Mar 2024
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance Omer Goldman Avi Caciularu Matan Eyal Kris Cao Idan Szpektor Reut Tsarfaty 51 23 0 10 Mar 2024
Online Adaptation of Language Models with a Memory of Amortized Contexts Jihoon Tack Jaehyung Kim Eric Mitchell Jinwoo Shin Yee Whye Teh Jonathan Richard Schwarz KELM 55 18 0 07 Mar 2024
Preference optimization of protein language models as a multi-objective binder design paradigm Pouria A. Mistani Venkatesh Mysore 45 6 0 07 Mar 2024
Did Translation Models Get More Robust Without Anyone Even Noticing? Ben Peters André F. T. Martins 44 3 0 06 Mar 2024
Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ Carolin Holtermann Paul Röttger Timm Dill Anne Lauscher ELM LRM 45 24 0 06 Mar 2024
General2Specialized LLMs Translation for E-commerce Kaidi Chen Ben Chen Dehong Gao Huangyu Dai Wen Jiang Wei Ning Shanqing Yu Libin Yang Xiaoyan Cai 17 8 0 06 Mar 2024
Breeze-7B Technical Report Chan-Jan Hsu Chang-Le Liu Feng-Ting Liao Po-Chun Hsu Yi-Chang Chen Da-Shan Shiu 34 2 0 05 Mar 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 68 81 0 05 Mar 2024
Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review Iryna Hartsock Ghulam Rasool 54 64 0 04 Mar 2024
A Generative Approach for Wikipedia-Scale Visual Entity Recognition Mathilde Caron Ahmet Iscen Alireza Fathi Cordelia Schmid 45 5 0 04 Mar 2024
Transformers for Low-Resource Languages:Is Féidir Linn! Séamus Lankford H. Alfi Tamás Sarlós 42 17 0 04 Mar 2024
adaptNMT: an open-source, language-agnostic development environment for Neural Machine Translation Séamus Lankford Haithem Afli Andy Way 34 3 0 04 Mar 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 47 3 0 04 Mar 2024
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation Heegon Jin Seonil Son Jemin Park Youngseok Kim Hyungjong Noh Yeonsoo Lee 41 2 0 03 Mar 2024
VBART: The Turkish LLM Meliksah Turker Mehmet Erdi Ari Aydin Han VLM 39 4 0 02 Mar 2024
Greed is All You Need: An Evaluation of Tokenizer Inference Methods Omri Uzan Craig W. Schmidt Chris Tanner Yuval Pinter 51 14 0 02 Mar 2024
Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models Jinbiao Yang LLMAG 105 11 0 01 Mar 2024
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models Frederik Kunstner Robin Yadav Alan Milligan Mark Schmidt Alberto Bietti 49 26 0 29 Feb 2024
Compact Speech Translation Models via Discrete Speech Units Pretraining Tsz Kin Lam Alexandra Birch Barry Haddow 66 2 0 29 Feb 2024
Beyond Language Models: Byte Models are Digital World Simulators Shangda Wu Xu Tan Zili Wang Rui Wang Xiaobing Li Maosong Sun 35 12 0 29 Feb 2024
EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation Yuqiao Wen Behzad Shayegh Chenyang Huang Yanshuai Cao Lili Mou 63 5 0 29 Feb 2024
Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translation Yusheng Liao Yanfeng Wang Yu Wang AI4CE 35 0 0 28 Feb 2024
Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models Ercong Nie Shuzhou Yuan Bolei Ma Helmut Schmid Michael Farber Frauke Kreuter Hinrich Schütze ReLM 99 6 0 28 Feb 2024
Tokenization Is More Than Compression Craig W. Schmidt Varshini Reddy Haoran Zhang Alec Alameddine Omri Uzan Yuval Pinter Chris Tanner 61 28 0 28 Feb 2024
Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey Dinh-Viet-Toan Le Louis Bigo Mikaela Keller Dorien Herremans MedIm 41 9 0 27 Feb 2024
Quantum linear algebra is all you need for Transformer architectures Naixu Guo Zhan Yu Matthew Choi Aman Agrawal Kouhei Nakaji Alán Aspuru-Guzik Patrick Rebentrost AI4CE 35 16 0 26 Feb 2024
Parameter-efficient Prompt Learning for 3D Point Cloud Understanding Hongyu Sun Yongcai Wang Wang Chen Haoran Deng Deying Li VPVLM 55 5 0 24 Feb 2024
Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR Jintao Jiang Yingbo Gao Mohammad Zeineldeen Zoltán Tüske 39 0 0 23 Feb 2024
DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators Xinglin Lyu Junhui Li Yanqing Zhao Min Zhang Daimeng Wei Shimin Tao Hao Yang Min Zhang 55 4 0 23 Feb 2024
How Important Is Tokenization in French Medical Masked Language Models? Yanis Labrak Adrien Bazoge B. Daille Mickael Rouvier Richard Dufour 44 1 0 22 Feb 2024
Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs Aaditya K. Singh DJ Strouse 46 46 0 22 Feb 2024
The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations Aina Garí Soler Matthieu Labeau Chloé Clavel VLM 47 2 0 22 Feb 2024
Two Counterexamples to Tokenization and the Noiseless Channel Marco Cognetta Vilém Zouhar Sangwhan Moon Naoaki Okazaki 32 0 0 22 Feb 2024
MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems Lichi Li Zainul Din Zhen Tan Sam London Tianlong Chen Ajay Daptardar 54 0 0 22 Feb 2024
Subobject-level Image Tokenization Delong Chen Samuel Cahyawijaya Jianfeng Liu Baoyuan Wang Pascale Fung VLM OCL 60 7 0 22 Feb 2024
UniCell: Universal Cell Nucleus Classification via Prompt Learning Junjia Huang Haofeng Li Xiang Wan Guanbin Li 42 0 0 20 Feb 2024
MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces Tianyu Zheng Ge Zhang Xingwei Qu Ming Kuang Stephen W. Huang Zhaofeng He OffRL 58 1 0 20 Feb 2024
Emergent Word Order Universals from Cognitively-Motivated Language Models Tatsuki Kuribayashi Ryo Ueda Ryosuke Yoshida Yohei Oseki Ted Briscoe Timothy Baldwin 46 2 0 19 Feb 2024
Text Diffusion with Reinforced Conditioning Yuxuan Liu Tianchi Yang Shaohan Huang Zihan Zhang Haizhen Huang Furu Wei Weiwei Deng Feng Sun Qi Zhang 35 1 0 19 Feb 2024
Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges Jiajia Wang Jimmy X. Huang Xinhui Tu Junmei Wang Angela J. Huang Md Tahmid Rahman Laskar Amran Bhuiyan 42 28 0 18 Feb 2024
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference Atsuki Yamaguchi Aline Villavicencio Nikolaos Aletras 32 7 0 16 Feb 2024
Conversational SimulMT: Efficient Simultaneous Translation with Large Language Models Minghan Wang Thuy-Trang Vu Yuxia Wang Ehsan Shareghi Gholamreza Haffari 48 2 0 16 Feb 2024
PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control Ruijie Zheng Ching-An Cheng Hal Daumé Furong Huang Andrey Kolobov 38 9 0 16 Feb 2024
Evaluating and Improving Continual Learning in Spoken Language Understanding Muqiao Yang Xiang Li Umberto Cappellazzo Shinji Watanabe Bhiksha Raj CLL 36 0 0 16 Feb 2024
Fast Vocabulary Transfer for Language Model Compression Leonidas Gee Andrea Zugarini Leonardo Rigutini Paolo Torroni 35 27 0 15 Feb 2024
Multi-word Tokenization for Sequence Compression Leonidas Gee Leonardo Rigutini Marco Ernandes Andrea Zugarini 18 8 0 15 Feb 2024
Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent Quentin Gallouedec E. Beeching Clément Romac Emmanuel Dellandrea 37 11 0 15 Feb 2024
Knowledge of Pretrained Language Models on Surface Information of Tokens Tatsuya Hiraoka Naoaki Okazaki 32 1 0 15 Feb 2024
Improving Non-autoregressive Machine Translation with Error Exposure and Consistency Regularization Xinran Chen Sufeng Duan Gongshen Liu 35 0 0 15 Feb 2024