Pretraining without Wordpieces: Learning Over a Vocabulary of Millions
of Words

Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words

International Journal of Machine Learning and Cybernetics (IJMLC), 2022

24 February 2022

ArXiv (abs)PDF HTML

Papers citing "Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words"

6 / 6 papers shown

Title
Towards understanding evolution of science through language model series Junjie Dong Zhuoqi Lyu Qing Ke AI4TS 326 0 0 15 Sep 2024
Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew Eylon Gueta Omer Goldman Reut Tsarfaty 107 3 0 01 Nov 2023
Biomedical Language Models are Robust to Sub-optimal TokenizationWorkshop on Biomedical Natural Language Processing (BioNLP), 2023 Bernal Jiménez Gutiérrez Huan Sun Yu-Chuan Su 135 8 0 30 Jun 2023
ESIE-BERT: Enriching Sub-words Information Explicitly with BERT for Joint Intent Classification and SlotFilling Yutian Guo Zhilong Xie Xingyan Chen Huangen Chen Leilei Wang Huaming Du Shaopeng Wei Yu Zhao Qing Li Ganglu Wu 270 15 0 27 Nov 2022
Word-Level Representation From Bytes For Language Modeling Chul Lee Qipeng Guo Xipeng Qiu 177 1 0 23 Nov 2022
Topic-Grained Text Representation-based Model for Document RetrievalInternational Conference on Artificial Neural Networks (ICANN), 2022 Mengxue Du Shasha Li Jie Yu Jun Ma Bing Ji Bin Ji Wuhang Lin Zibo Yi 90 3 0 11 Jul 2022