Neural Machine Translation of Rare Words with Subword Units

31 August 2015

Papers citing "Neural Machine Translation of Rare Words with Subword Units"

50 / 3,808 papers shown

Title
Counting Ability of Large Language Models and Impact of Tokenization Xiang Zhang Juntai Cao Chenyu You LRM 40 5 0 25 Oct 2024
Bielik 7B v0.1: A Polish Language Model -- Development, Insights, and Evaluation Krzysztof Ociepa Łukasz Flis Krzysztof Wróbel Adrian Gwoździej Remigiusz Kinas 27 1 0 24 Oct 2024
Subword Embedding from Bytes Gains Privacy without Sacrificing Accuracy and Complexity Mengjiao Zhang Jia Xu FedML 24 0 0 21 Oct 2024
Tokenization as Finite-State Transduction Marco Cognetta Naoaki Okazaki 26 0 0 21 Oct 2024
Moonshine: Speech Recognition for Live Transcription and Voice Commands Nat Jeffries Evan King M. Kudlur Guy Nicholson James Wang Pete Warden 39 5 0 21 Oct 2024
Action abstractions for amortized sampling Oussama Boussif Léna Néhale Ezzine J. Viviano Michał Koziarski Moksh Jain Nikolay Malkin Emmanuel Bengio Rim Assouel Yoshua Bengio 28 0 0 19 Oct 2024
LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems Nan Xu Xuezhe Ma LRM 59 3 0 18 Oct 2024
Quantity vs. Quality of Monolingual Source Data in Automatic Text Translation: Can It Be Too Little If It Is Too Good? Idris Abdulmumin B. Galadanci G. Aliyu Shamsuddeen Hassan Muhammad 37 1 0 17 Oct 2024
Representation Learning of Structured Data for Medical Foundation Models Vijay Prakash Dwivedi Viktor Schlegel Andy T. Liu Thanh-Tung Nguyen Abhinav Ramesh Kashyap Jeng Wei Wei-Hsian Yin Stefan Winkler R. Tan 35 0 0 17 Oct 2024
Prompt Compression for Large Language Models: A Survey Zongqian Li Yinhong Liu Yixuan Su Nigel Collier MQ 55 11 0 16 Oct 2024
Evaluating Morphological Compositional Generalization in Large Language Models Mete Ismayilzada Yuan Chiang Jonne Sälevä Hale Sirin Abdullatif Köksal Bhuwan Dhingra Antoine Bosselut Lonneke van der Plas Duygu Ataman 33 2 0 16 Oct 2024
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models Kushal Tatariya Vladimir Araujo Thomas Bauwens Miryam de Lhoneux VLM 38 0 0 15 Oct 2024
The Fair Language Model Paradox Andrea Pinto Tomer Galanti Randall Balestriero 25 0 0 15 Oct 2024
Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5 Thao Anh Dang Limor Raviv Lukas Galke 25 1 0 15 Oct 2024
Latent Action Pretraining from Videos Seonghyeon Ye Joel Jang Byeongguk Jeon Sejune Joo Jianwei Yang ... Kimin Lee J. Gao Luke Zettlemoyer Dieter Fox Minjoon Seo 35 28 0 15 Oct 2024
Generalized Adversarial Code-Suggestions: Exploiting Contexts of LLM-based Code-Completion Karl Rubel Maximilian Noppel Christian Wressnegger AAML SILM 28 0 0 14 Oct 2024
Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification? Gabriel Roccabruna Massimo Rizzoli Giuseppe Riccardi 31 1 0 14 Oct 2024
ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-training Zhouqiang Jiang Bowen Wang Junhao Chen Yuta Nakashima 30 2 0 14 Oct 2024
4-LEGS: 4D Language Embedded Gaussian Splatting Gal Fiebelman Tamir Cohen Ayellet Morgenstern Peter Hedman Hadar Averbuch-Elor 3DGS 46 3 0 14 Oct 2024
An Annotated Dataset of Errors in Premodern Greek and Baselines for Detecting Them Creston Brooks J. Haubold Charlie Cowen-Breen Jay White Desmond DeVaul Frederick Riemenschneider Karthik Narasimhan B. Graziosi 30 0 0 14 Oct 2024
Encoding Agent Trajectories as Representations with Sequence Transformers Athanasios Tsiligkaridis Nicholas Kalinowski Zhongheng Li Elizabeth Hou 31 1 0 11 Oct 2024
Generation with Dynamic Vocabulary Yanting Liu Tao Ji Changzhi Sun Yuanbin Wu Xiaoling Wang 45 0 0 11 Oct 2024
The Large Language Model GreekLegalRoBERTa Vasileios Saketos D. Pantazi Manolis Koubarakis AILaw 34 0 0 10 Oct 2024
Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow Cyrile Delestre Yoann Sola 34 0 0 10 Oct 2024
Generative Model for Less-Resourced Language with 1 billion parameters Domen Vreš Martin Božič Aljaž Potočnik Tomaž Martinčič Marko Robnik-Šikonja 26 1 0 09 Oct 2024
Inference over Unseen Entities, Relations and Literals on Knowledge Graphs Caglar Demir N'Dah Jean Kouagou Arnab Sharma Axel-Cyrille Ngonga Ngomo 28 0 0 09 Oct 2024
OrionNav: Online Planning for Robot Autonomy with Context-Aware LLM and Open-Vocabulary Semantic Scene Graphs Venkata Naren Devarakonda Raktim Gautam Goswami Ali Umut Kaypak Naman Patel Rooholla Khorrambakht Prashanth Krishnamurthy Farshad Khorrami LM&Ro 39 3 0 08 Oct 2024
Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning Alexis Allemann Àlex R. Atrio Andrei Popescu-Belis 31 0 0 08 Oct 2024
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 52 12 0 08 Oct 2024
Neural machine translation system for Lezgian, Russian and Azerbaijani languages Alidar Asvarov Andrey Grabovoy 37 0 0 07 Oct 2024
Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics Stefano Perrella Lorenzo Proietti Pere-Lluís Huguet Cabot Edoardo Barba Roberto Navigli 26 3 0 07 Oct 2024
HALL-E: Hierarchical Neural Codec Language Model for Minute-Long Zero-Shot Text-to-Speech Synthesis Yuto Nishimura Takumi Hirose Masanari Ohi Hideki Nakayama Nakamasa Inoue VLM 31 1 0 06 Oct 2024
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks Alex Cloud Jacob Goldman-Wetzler Evžen Wybitul Joseph Miller Alexander Matt Turner 36 4 0 06 Oct 2024
Efficient and Robust Long-Form Speech Recognition with Hybrid H3-Conformer Tomoki Honda S. Sakai Tatsuya Kawahara 28 0 0 05 Oct 2024
Toxic Subword Pruning for Dialogue Response Generation on Large Language Models Hongyuan Lu Wai Lam 17 0 0 05 Oct 2024
Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT? Àlex R. Atrio Alexis Allemann Ljiljana Dolamic Andrei Popescu-Belis 38 1 0 05 Oct 2024
MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions Yekun Chai Haoran Sun Huang Fang Shuohuan Wang Yu Sun Hua Wu 198 1 0 03 Oct 2024
From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities Wanpeng Zhang Zilong Xie Yicheng Feng Yijiang Li Xingrun Xing Sipeng Zheng Zongqing Lu MLLM 30 0 0 03 Oct 2024
HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR Hainan Xu Travis M. Bartley Vladimir Bataev Boris Ginsburg 202 0 0 03 Oct 2024
Large Language Models as Markov Chains Oussama Zekri Ambroise Odonnat Abdelhakim Benechehab Linus Bleistein Nicolas Boullé I. Redko 48 10 0 03 Oct 2024
Mind Scramble: Unveiling Large Language Model Psychology Via Typoglycemia Miao Yu Junyuan Mao Guibin Zhang Jingheng Ye Junfeng Fang Aoxiao Zhong Yang Liu Yuxuan Liang Kun Wang Qingsong Wen 44 2 0 02 Oct 2024
Analyzing Byte-Pair Encoding on Monophonic and Polyphonic Symbolic Music: A Focus on Musical Phrase Segmentation Dinh-Viet-Toan Le Louis Bigo Mikaela Keller 34 1 0 02 Oct 2024
The Conformer Encoder May Reverse the Time Dimension Robin Schmitt Albert Zeyer Mohammad Zeineldeen Ralf Schluter Hermann Ney 36 0 0 01 Oct 2024
Boosting Hybrid Autoregressive Transducer-based ASR with Internal Acoustic Model Training and Dual Blank Thresholding Takafumi Moriya Takanori Ashihara Masato Mimura Hiroshi Sato Kohei Matsuura Ryo Masumura Taichi Asami 24 0 0 30 Sep 2024
Enhancing High-order Interaction Awareness in LLM-based Recommender Model Xinfeng Wang Jin Cui Fumiyo Fukumoto Yoshimi Suzuki 30 3 0 30 Sep 2024
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling David Grangier Simin Fan Skyler Seto Pierre Ablin 44 3 0 30 Sep 2024
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference Yejin Lee Anna Y. Sun Basil Hosmer Bilge Acun Can Balioglu ... Ram Pasunuru Scott Yih Sravya Popuri Xing Liu Carole-Jean Wu 57 2 0 30 Sep 2024
When Molecular GAN Meets Byte-Pair Encoding Huidong Tang Chen Li Yasuhiko Morimoto 42 0 0 29 Sep 2024
Performance Evaluation of Tokenizers in Large Language Models for the Assamese Language Sagar Tamang Dibya Jyoti Bora 41 3 0 28 Sep 2024
Exploring Language Model Generalization in Low-Resource Extractive QA Saptarshi Sengupta Wenpeng Yin Preslav Nakov Shreya Ghosh Suhang Wang 27 0 0 27 Sep 2024