ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1808.06226
  4. Cited By
SentencePiece: A simple and language independent subword tokenizer and
  detokenizer for Neural Text Processing

SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

19 August 2018
Taku Kudo
John Richardson
ArXiv (abs)PDFHTMLGithub (10925★)

Papers citing "SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing"

50 / 2,064 papers shown
The Linear Representation Hypothesis and the Geometry of Large Language
  Models
The Linear Representation Hypothesis and the Geometry of Large Language ModelsInternational Conference on Machine Learning (ICML), 2023
Kiho Park
Yo Joong Choe
Victor Veitch
LLMSVMILM
467
322
0
07 Nov 2023
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoEInternational Conference on Learning Representations (ICLR), 2023
Zeren Chen
Ziqin Wang
Zhen Wang
Huayang Liu
Zhen-fei Yin
Si Liu
Lu Sheng
Wanli Ouyang
Yu Qiao
Jing Shao
MoE
268
17
0
05 Nov 2023
Too Much Information: Keeping Training Simple for BabyLMs
Too Much Information: Keeping Training Simple for BabyLMs
Lukas Edman
Lisa Bylinina
192
6
0
03 Nov 2023
Server-side Rescoring of Spoken Entity-centric Knowledge Queries for
  Virtual Assistants
Server-side Rescoring of Spoken Entity-centric Knowledge Queries for Virtual AssistantsInternational Journal of Speech Technology (IJST), 2023
Youyuan Zhang
Sashank Gondala
Thiago Fraga-Silva
Christophe Van Gysel
259
3
0
02 Nov 2023
ACES: Translation Accuracy Challenge Sets at WMT 2023
ACES: Translation Accuracy Challenge Sets at WMT 2023Conference on Machine Translation (WMT), 2023
Chantal Amrhein
Nikita Moghe
Liane Guillou
ELM
186
4
0
02 Nov 2023
From Image to Language: A Critical Analysis of Visual Question Answering
  (VQA) Approaches, Challenges, and Opportunities
From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and OpportunitiesInformation Fusion (Inf. Fusion), 2023
Md Farhan Ishmam
Md Sakib Hossain Shovon
M. F. Mridha
Nilanjan Dey
399
71
0
01 Nov 2023
The Unreasonable Effectiveness of Random Target Embeddings for
  Continuous-Output Neural Machine Translation
The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine TranslationNorth American Chapter of the Association for Computational Linguistics (NAACL), 2023
Evgeniia Tokarchuk
Vlad Niculae
183
2
0
31 Oct 2023
Towards a Deep Understanding of Multilingual End-to-End Speech
  Translation
Towards a Deep Understanding of Multilingual End-to-End Speech TranslationConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Haoran Sun
Xiaohu Zhao
Yikun Lei
Shaolin Zhu
Deyi Xiong
209
8
0
31 Oct 2023
Is Robustness Transferable across Languages in Multilingual Neural
  Machine Translation?
Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Leiyu Pan
Supryadi Supryadi
Deyi Xiong
AAML
287
0
0
31 Oct 2023
CreoleVal: Multilingual Multitask Benchmarks for Creoles
CreoleVal: Multilingual Multitask Benchmarks for CreolesTransactions of the Association for Computational Linguistics (TACL), 2023
Heather Lent
Kushal Tatariya
Mary Dabre
Yiyi Chen
Marcell Richard Fekete
...
Miryam de Lhoneux
Daniel Hershcovich
Michel DeGraff
Anders Sogaard
Johannes Bjerva
SLR
352
15
0
30 Oct 2023
Skywork: A More Open Bilingual Foundation Model
Skywork: A More Open Bilingual Foundation Model
Tianwen Wei
Liang Zhao
Lichang Zhang
Bo Zhu
Lijie Wang
...
Yongyi Peng
Xiaojuan Liang
Shuicheng Yan
Han Fang
Yahui Zhou
275
121
0
30 Oct 2023
Roles of Scaling and Instruction Tuning in Language Perception: Model
  vs. Human Attention
Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human AttentionConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Changjiang Gao
Shujian Huang
Jixing Li
Jiajun Chen
LRMALM
357
9
0
29 Oct 2023
Probing LLMs for Joint Encoding of Linguistic Categories
Probing LLMs for Joint Encoding of Linguistic CategoriesConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Giulio Starace
Konstantinos Papakostas
Rochelle Choenni
Apostolos Panagiotopoulos
Matteo Rosati
Alina Leidinger
Ekaterina Shutova
258
13
0
28 Oct 2023
Unified Segment-to-Segment Framework for Simultaneous Sequence
  Generation
Unified Segment-to-Segment Framework for Simultaneous Sequence GenerationNeural Information Processing Systems (NeurIPS), 2023
Shaolei Zhang
Yang Feng
260
9
0
27 Oct 2023
Lil-Bevo: Explorations of Strategies for Training Language Models in
  More Humanlike Ways
Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
Venkata S Govindarajan
Juan Diego Rodriguez
Kaj Bostrom
Kyle Mahowald
299
1
0
26 Oct 2023
Learning to Abstract with Nonparametric Variational Information
  Bottleneck
Learning to Abstract with Nonparametric Variational Information BottleneckConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Melika Behjati
Fabio Fehr
James Henderson
SSL
221
4
0
26 Oct 2023
EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual
  Representation Learning
EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation LearningNeural Information Processing Systems (NeurIPS), 2023
Ping Guo
Xiangpeng Wei
Yue Hu
Baosong Yang
Dayiheng Liu
Fei Huang
Jun Xie
250
3
0
26 Oct 2023
CL-MASR: A Continual Learning Benchmark for Multilingual ASR
CL-MASR: A Continual Learning Benchmark for Multilingual ASRIEEE/ACM Transactions on Audio Speech and Language Processing (TASLP), 2023
Luca Della Libera
Pooneh Mousavi
Salah Zaiem
Cem Subakan
Mirco Ravanelli
AuLLMCLL
266
16
0
25 Oct 2023
Enhanced Simultaneous Machine Translation with Word-level Policies
Enhanced Simultaneous Machine Translation with Word-level PoliciesConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Kang Kim
Hankyu Cho
234
3
0
25 Oct 2023
Samsung R&D Institute Philippines at WMT 2023
Samsung R&D Institute Philippines at WMT 2023Conference on Machine Translation (WMT), 2023
Jan Christian Blaise Cruz
151
6
0
25 Oct 2023
MindLLM: Pre-training Lightweight Large Language Model from Scratch,
  Evaluations and Domain Applications
MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications
Yizhe Yang
Huashan Sun
Jiawei Li
Runheng Liu
Yinghao Li
Yuhang Liu
Heyan Huang
Yang Gao
ALMLRM
187
14
0
24 Oct 2023
A Joint Matrix Factorization Analysis of Multilingual Representations
A Joint Matrix Factorization Analysis of Multilingual RepresentationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Zheng Zhao
Yftah Ziser
Bonnie Webber
Shay B. Cohen
254
5
0
24 Oct 2023
Leveraging Timestamp Information for Serialized Joint Streaming
  Recognition and Translation
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and TranslationIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023
Sara Papi
Peidong Wang
Junkun Chen
Jian Xue
Naoyuki Kanda
Jinyu Li
Yashesh Gaur
142
4
0
23 Oct 2023
Code-Switching with Word Senses for Pretraining in Neural Machine
  Translation
Code-Switching with Word Senses for Pretraining in Neural Machine TranslationConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Vivek Iyer
Edoardo Barba
Alexandra Birch
Jeff Z. Pan
Roberto Navigli
243
3
0
21 Oct 2023
Ask Language Model to Clean Your Noisy Translation Data
Ask Language Model to Clean Your Noisy Translation Data
Quinten Bolding
Baohao Liao
Brandon James Denis
Jun Luo
Christof Monz
219
9
0
20 Oct 2023
The CHiME-7 Challenge: System Description and Performance of NeMo Team's
  DASR System
The CHiME-7 Challenge: System Description and Performance of NeMo Team's DASR System
T. Park
He Huang
Ante Jukić
Kunal Dhawan
Krishna C. Puvvada
Nithin Rao Koluguri
Nikolay Karpov
A. Laptev
Jagadeesh Balam
Boris Ginsburg
200
11
0
18 Oct 2023
Direct Neural Machine Translation with Task-level Mixture of Experts
  models
Direct Neural Machine Translation with Task-level Mixture of Experts models
Isidora Chara Tourni
Subhajit Naskar
MoE
220
0
0
18 Oct 2023
SPEED: Speculative Pipelined Execution for Efficient Decoding
SPEED: Speculative Pipelined Execution for Efficient Decoding
Coleman Hooper
Sehoon Kim
Hiva Mohammadzadeh
Hasan Genç
Kurt Keutzer
A. Gholami
Y. Shao
204
48
0
18 Oct 2023
BUT CHiME-7 system description
BUT CHiME-7 system description
M. Karafiát
Karel Veselý
Igor Szöke
Ladislav Mošner
Karel Beneš
Marcin Witkowski
Germán Barchi
L. Pepino
135
2
0
18 Oct 2023
ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency
  by Automatic Task Formation
ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation
Jaap Jumelet
Michael Hanna
Marianne de Heer Kloots
Anna Langedijk
Charlotte Pouw
Oskar van der Wal
199
3
0
17 Oct 2023
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text
  Processing
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text ProcessingConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Quoc-Nam Nguyen
Thang Chau Phan
Duc-Vu Nguyen
Kiet Van Nguyen
226
14
0
17 Oct 2023
IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing
  Interactive Machine Translation Systems
IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation SystemsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Xu Huang
Zhirui Zhang
Ruize Gao
Yichao Du
Lemao Liu
Gouping Huang
Shuming Shi
Jiajun Chen
Shujian Huang
VLM
111
0
0
17 Oct 2023
Iterative Shallow Fusion of Backward Language Model for End-to-End
  Speech Recognition
Iterative Shallow Fusion of Backward Language Model for End-to-End Speech RecognitionIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023
A. Ogawa
Takafumi Moriya
Naoyuki Kamo
Naohiro Tawara
Marc Delcroix
152
3
0
17 Oct 2023
Approximating Two-Layer Feedforward Networks for Efficient Transformers
Approximating Two-Layer Feedforward Networks for Efficient TransformersConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Róbert Csordás
Kazuki Irie
Jürgen Schmidhuber
MoE
414
23
0
16 Oct 2023
Towards a Better Understanding of Variations in Zero-Shot Neural Machine
  Translation Performance
Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation PerformanceConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Shaomu Tan
Christof Monz
353
15
0
16 Oct 2023
Optimized Tokenization for Transcribed Error Correction
Optimized Tokenization for Transcribed Error CorrectionConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Tomer Wullach
Shlomo E. Chazan
198
0
0
16 Oct 2023
Prediction of Arabic Legal Rulings using Large Language Models
Prediction of Arabic Legal Rulings using Large Language Models
Adel Ammar
Anis Koubaa
Bilel Benjdira
Omar Najar
Serry Sibaee
AILawELM
218
18
0
16 Oct 2023
End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder
  and Input Feature Analysis
End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis
Can Cui
Imran A. Sheikh
Mostafa Sadeghi
Emmanuel Vincent
167
5
0
16 Oct 2023
Personalization of CTC-based End-to-End Speech Recognition Using
  Pronunciation-Driven Subword Tokenization
Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenization
Zhihong Lei
Ernest Pusateri
Shiyi Han
Leo Liu
Mingbin Xu
...
R. Travadi
Youyuan Zhang
Mirko Hannemann
Man-Hung Siu
Zhen Huang
211
10
0
16 Oct 2023
UvA-MT's Participation in the WMT23 General Translation Shared Task
UvA-MT's Participation in the WMT23 General Translation Shared Task
Di Wu
Shaomu Tan
David Stap
Ali Araabi
Christof Monz
216
4
0
15 Oct 2023
Generative Adversarial Training for Text-to-Speech Synthesis Based on
  Raw Phonetic Input and Explicit Prosody Modelling
Generative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modelling
Tiberiu Boros
Stefan Daniel Dumitrescu
Ionut Mironica
Radu Chivereanu
GAN
152
2
0
14 Oct 2023
Embarrassingly Simple Text Watermarks
Embarrassingly Simple Text Watermarks
Ryoma Sato
Yuki Takezawa
Han Bao
Kenta Niwa
Makoto Yamada
WaLM
322
24
0
13 Oct 2023
Tokenizer Choice For LLM Training: Negligible or Crucial?
Tokenizer Choice For LLM Training: Negligible or Crucial?
Mehdi Ali
Michael Fromm
Klaudia Thellmann
Richard Rutmann
Max Lübbering
...
Malte Ostendorff
Samuel Weinbach
R. Sifa
Stefan Kesselheim
Nicolas Flores-Herr
562
102
0
12 Oct 2023
Toward Joint Language Modeling for Speech Units and Text
Toward Joint Language Modeling for Speech Units and TextConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Ju-Chieh Chou
Chung-Ming Chien
Wei-Ning Hsu
Karen Livescu
Arun Babu
Alexis Conneau
Alexei Baevski
Michael Auli
VLM
233
27
0
12 Oct 2023
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
InstructRetro: Instruction Tuning post Retrieval-Augmented PretrainingInternational Conference on Machine Learning (ICML), 2023
Wei Ping
Ming-Yu Liu
Lawrence C. McAfee
Peng Xu
Bo Li
Mohammad Shoeybi
Bryan Catanzaro
RALM
468
69
0
11 Oct 2023
MatFormer: Nested Transformer for Elastic Inference
MatFormer: Nested Transformer for Elastic InferenceNeural Information Processing Systems (NeurIPS), 2023
Devvrit
Sneha Kudugunta
Aditya Kusupati
Tim Dettmers
Kaifeng Chen
...
Yulia Tsvetkov
Hannaneh Hajishirzi
Sham Kakade
Ali Farhadi
Prateek Jain
255
61
0
11 Oct 2023
An Empirical Study of Instruction-tuning Large Language Models in
  Chinese
An Empirical Study of Instruction-tuning Large Language Models in ChineseConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Q. Si
Tong Wang
Zheng Lin
Xu Zhang
Yanan Cao
Weiping Wang
ALM
199
23
0
11 Oct 2023
On the Impact of Cross-Domain Data on German Language Models
On the Impact of Cross-Domain Data on German Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Amin Dada
Aokun Chen
C.A.I. Peng
Kaleb E. Smith
Ahmad Idrissi-Yaghir
...
Daniel Truhn
Jan Egger
Jiang Bian
Jens Kleesiek
Yonghui Wu
188
7
0
11 Oct 2023
BioT5: Enriching Cross-modal Integration in Biology with Chemical
  Knowledge and Natural Language Associations
BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language AssociationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Qizhi Pei
Wei Zhang
Jinhua Zhu
Kehan Wu
Ran Bi
Lijun Wu
Ziheng Lu
Rui Yan
321
106
0
11 Oct 2023
Acoustic Model Fusion for End-to-end Speech Recognition
Acoustic Model Fusion for End-to-end Speech RecognitionAutomatic Speech Recognition & Understanding (ASRU), 2023
Zhihong Lei
Mingbin Xu
Shiyi Han
Leo Liu
Zhen Huang
...
Yuanyuan Zhang
Ernest Pusateri
Mirko Hannemann
Yaqiao Deng
Man-Hung Siu
209
6
0
10 Oct 2023
Previous
123...131415...404142
Next