Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1808.06226
Cited By
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing
19 August 2018
Taku Kudo
John Richardson
Re-assign community
ArXiv (abs)
PDF
HTML
Github (10925★)
Papers citing
"SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing"
50 / 2,064 papers shown
Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition
Vahid Noroozi
Somshubra Majumdar
Ankur Kumar
Jagadeesh Balam
Boris Ginsburg
441
22
0
27 Dec 2023
PanGu-
π
π
π
: Enhancing Language Model Architectures via Nonlinearity Compensation
Yunhe Wang
Hanting Chen
Yehui Tang
Tianyu Guo
Kai Han
...
Qinghua Xu
Qun Liu
Jun Yao
Chao Xu
Dacheng Tao
285
24
0
27 Dec 2023
Gemini Pro Defeated by GPT-4V: Evidence from Education
Gyeong-Geon Lee
Ehsan Latif
Lehong Shi
Xiaoming Zhai
257
34
0
27 Dec 2023
Dotless Representation of Arabic Text: Analysis and Modeling
Maged S. Al-Shaibani
Irfan Ahmad
173
1
0
26 Dec 2023
PersianLLaMA: Towards Building First Persian Large Language Model
Mohammad Amin Abbasi
A. Ghafouri
Mahdi Firouzmandi
Hassan Naderi
B. Minaei-Bidgoli
253
17
0
25 Dec 2023
YAYI 2: Multilingual Open-Source Large Language Models
Yin Luo
Qingchao Kong
Nan Xu
Jia Cao
Bao Hao
...
Zhaoxin Yu
Zhengda Luo
Wenji Mao
Lei Wang
Dajun Zeng
ALM
OSLM
169
7
0
22 Dec 2023
Typhoon: Thai Large Language Models
Kunat Pipatanakul
Phatrasek Jirabovonvisut
Potsawee Manakul
Sittipong Sripaisarnmongkol
Ruangsak Patomwong
Pathomporn Chokchainant
Kasima Tharnpipitchai
209
32
0
21 Dec 2023
Soft Alignment of Modality Space for End-to-end Speech Translation
Yuhao Zhang
Kaiqi Kou
Bei Li
Chen Xu
Chunliang Zhang
Tong Xiao
Jingbo Zhu
282
8
0
18 Dec 2023
Binary Code Summarization: Benchmarking ChatGPT/GPT-4 and Other Large Language Models
Xin Jin
Jonathan Larson
Weiwei Yang
Zhiqiang Lin
ELM
162
34
0
15 Dec 2023
VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation
Jinguo Zhu
Xiaohan Ding
Yixiao Ge
Yuying Ge
Sijie Zhao
Hengshuang Zhao
Xiaohua Wang
Ying Shan
ViT
VLM
188
46
0
14 Dec 2023
Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models
European Conference on Computer Vision (ECCV), 2023
Zhiyuan You
Zheyuan Li
Jinjin Gu
Zhenfei Yin
Tianfan Xue
Chao Dong
EGVM
399
90
0
14 Dec 2023
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
Neural Information Processing Systems (NeurIPS), 2023
Róbert Csordás
Piotr Piekos
Kazuki Irie
Jürgen Schmidhuber
MoE
227
27
0
13 Dec 2023
N-Gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding
AAAI Conference on Artificial Intelligence (AAAI), 2023
Jinhao Tian
Zuchao Li
Jiajia Li
Ping Wang
327
6
0
13 Dec 2023
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception
Computer Vision and Pattern Recognition (CVPR), 2023
Yiran Qin
Enshen Zhou
Qichang Liu
Zhen-fei Yin
Lu Sheng
Ruimao Zhang
Yu Qiao
Jing Shao
LM&Ro
358
76
0
12 Dec 2023
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning
Dami Choi
Derrick Xin
Hamid Dadkhahi
Justin Gilmer
Ankush Garg
Orhan Firat
Chih-Kuan Yeh
Andrew M. Dai
Behrooz Ghorbani
280
6
0
11 Dec 2023
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
Hakan Inan
Kartikeya Upasani
Jianfeng Chi
Rashi Rungta
Krithika Iyer
...
Michael Tontchev
Qing Hu
Brian Fuller
Davide Testuggine
Madian Khabsa
AI4MH
437
750
0
07 Dec 2023
Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition
Yukiya Hono
Koh Mitsuda
Tianyu Zhao
Kentaro Mitsui
Toshiaki Wakatsuki
Kei Sawada
AuLLM
256
16
0
06 Dec 2023
Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpus
Automatic Speech Recognition & Understanding (ASRU), 2023
Yi-Hui Chou
Kalvin Chang
Meng-Ju Wu
Winston Ou
Alice Wen-Hsin Bi
...
Iu-Tshian Phoann
Winnie Chang
Chenxuan Cui
Noel Chen
Jiatong Shi
182
6
0
06 Dec 2023
Large Language Models on Graphs: A Comprehensive Survey
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2023
Sara Szymkuć
Gang Liu
Chi Han
Meng Jiang
Heng Ji
Jiawei Han
AI4CE
339
249
0
05 Dec 2023
A Machine Learning Approach Towards SKILL Code Autocompletion
Enrique Dehaerne
Bappaditya Dey
Wannes Meert
195
0
0
04 Dec 2023
Using Large Language Models to Accelerate Communication for Users with Severe Motor Impairments
Shanqing Cai
Subhashini Venugopalan
Katie Seaver
Xiang Xiao
Katrin Tomanek
...
Daniel E Vance
Blair Casey
Steve M. Gleason
Philip Q. Nelson
Michael P. Brenner
246
10
0
03 Dec 2023
On Significance of Subword tokenization for Low Resource and Efficient Named Entity Recognition: A case study in Marathi
Harsh Chaudhari
A. Patil
Dhanashree Lavekar
Pranav Khairnar
Raviraj Joshi
Sachin Pande
163
0
0
03 Dec 2023
INarIG: Iterative Non-autoregressive Instruct Generation Model For Word-Level Auto Completion
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Hengchao Shang
Zongyao Li
Daimeng Wei
Jiaxin Guo
Minghan Wang
Xiaoyu Chen
Lizhi Lei
Hao Yang
187
0
0
30 Nov 2023
Leveraging VLM-Based Pipelines to Annotate 3D Objects
International Conference on Machine Learning (ICML), 2023
Rishabh Kabra
Loic Matthey
Alexander Lerchner
Niloy J. Mitra
274
10
0
29 Nov 2023
YUAN 2.0: A Large Language Model with Localized Filtering-based Attention
Shaohua Wu
Xudong Zhao
Shenling Wang
Jiangang Luo
Lingjun Li
...
Wei Wang
Tong Yu
Rongguo Zhang
Jiahua Zhang
Chao Wang
OSLM
490
7
0
27 Nov 2023
Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context
Elijah Matthew Rippeth
Marine Carpuat
Kevin Duh
Matt Post
149
2
0
27 Nov 2023
Learning to Skip for Language Modeling
Dewen Zeng
Nan Du
Tao Wang
Yuanzhong Xu
Tao Lei
Zhifeng Chen
Claire Cui
195
16
0
26 Nov 2023
OpusCleaner and OpusTrainer, open source toolkits for training Machine Translation and Large language models
Nikolay Bogoychev
Jelmer van der Linde
Graeme Nail
Barry Haddow
Jaume Zaragoza-Bernabeu
Gema Ramírez-Sánchez
Lukas Weymann
Tudor N. Mateiu
Jindvrich Helcl
Mikko Aulamo
VLM
161
1
0
24 Nov 2023
Machine Translation for Geéz Language
A. Wassie
223
6
0
24 Nov 2023
PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords
ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 2023
Panyut Sriwirote
Jalinee Thapiang
Vasan Timtong
Attapol T. Rutherford
185
8
0
21 Nov 2023
Multi-teacher Distillation for Multilingual Spelling Correction
Jingfen Zhang
Xuan Guo
S. Bodapati
Christopher Potts
KELM
157
4
0
20 Nov 2023
An Embodied Generalist Agent in 3D World
Jiangyong Huang
Silong Yong
Xiaojian Ma
Xiongkun Linghu
Puhao Li
Yan Wang
Qing Li
Song-Chun Zhu
Baoxiong Jia
Siyuan Huang
LM&Ro
340
296
0
18 Nov 2023
JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing
Shester Gueuwou
Sophie Siake
Colin Leong
Mathias Müller
SLR
322
21
0
16 Nov 2023
WatME: Towards Lossless Watermarking Through Lexical Redundancy
Liang Chen
Yatao Bian
Yang Deng
Deng Cai
Shuaiyi Li
Peilin Zhao
Kam-Fai Wong
WaLM
336
21
0
16 Nov 2023
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Tyler A. Chang
Catherine Arnett
Zhuowen Tu
Benjamin Bergen
LRM
343
25
0
15 Nov 2023
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
J. Michaelov
Catherine Arnett
Tyler A. Chang
Benjamin Bergen
190
19
0
15 Nov 2023
Memory Augmented Language Models through Mixture of Word Experts
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Cicero Nogueira dos Santos
James Lee-Thorp
Isaac Noble
Chung-Ching Chang
David C. Uthus
MoE
227
9
0
15 Nov 2023
How Vocabulary Sharing Facilitates Multilingualism in LLaMA?
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Fei Yuan
Shuai Yuan
Zhiyong Wu
Lei Li
316
16
0
15 Nov 2023
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
Yihong Liu
Peiqin Lin
Mingyang Wang
Hinrich Schütze
232
36
0
15 Nov 2023
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
Chenxi Whitehouse
Fantine Huot
Jasmijn Bastings
Mostafa Dehghani
Chu-Cheng Lin
Mirella Lapata
270
14
0
14 Nov 2023
Retrieve and Copy: Scaling ASR Personalization to Large Catalogs
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Sai Muralidhar Jayanthi
Devang Kulshreshtha
Saket Dingliwal
S. Ronanki
S. Bodapati
214
9
0
14 Nov 2023
On-the-Fly Fusion of Large Language Models and Machine Translation
Hieu T. Hoang
Huda Khayrallah
Marcin Junczys-Dowmunt
277
5
0
14 Nov 2023
Learning Mutually Informed Representations for Characters and Subwords
Yilin Wang
Xinyi Hu
Matthew R. Gormley
204
0
0
14 Nov 2023
Extending Multilingual Machine Translation through Imitation Learning
Wen Lai
Viktor Hangya
Kangyang Luo
Alexander Fraser
LRM
CLL
470
5
0
14 Nov 2023
Context Consistency between Training and Testing in Simultaneous Machine Translation
M. Zhong
Lemao Liu
Kehai Chen
Mingming Yang
Min Zhang
LRM
214
0
0
13 Nov 2023
Towards the Law of Capacity Gap in Distilling Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Chen Zhang
Qiuchi Li
Dawei Song
Zheyu Ye
Yan Gao
Yan Hu
ELM
380
32
0
13 Nov 2023
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Yassir Fathullah
Chunyang Wu
Egor Lakomkin
Ke Li
Junteng Jia
Shangguan Yuan
Jay Mahadeokar
Ozlem Kalinli
Christian Fuegen
Michael Seltzer
LM&MA
MLLM
AuLLM
270
64
0
12 Nov 2023
ReactionT5: a large-scale pre-trained model towards application of limited reaction data
Tatsuya Sagawa
Ryosuke Kojima
AI4CE
207
12
0
12 Nov 2023
Tamil-Llama: A New Tamil Language Model Based on Llama 2
Abhinand Balachandran
155
38
0
10 Nov 2023
Proceedings of the 5th International Workshop on Reading Music Systems
Jorge Calvo-Zaragoza
Alexander Pacha
Elona Shatri
113
0
0
07 Nov 2023
Previous
1
2
3
...
12
13
14
...
40
41
42
Next
