Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

23 June 2021

Zhen Qin

Papers citing "Charformer: Fast Character Transformers via Gradient-based Subword Tokenization"

50 / 50 papers shown

Title
Token-free Models for Sarcasm Detection Sumit Mamtani Maitreya Sonawane Kanika Agarwal Nishanth Sanjeev 36 0 0 02 May 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 145 0 0 25 Mar 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 3 0 17 Mar 2025
FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation Andrew Kiruluta Eric Lundy Andreas Lemos AI4TS 39 0 0 04 Mar 2025
Tokenization is Sensitive to Language Variation Anna Wegmann Dong Nguyen David Jurgens 77 1 0 24 Feb 2025
Digital Guardians: Can GPT-4, Perspective API, and Moderation API reliably detect hate speech in reader comments of German online newspapers? Manuel Weber Moritz Huber Maximilian Auch Alexander Döschl Max-Emanuel Keller P. Mandl 32 0 0 03 Jan 2025
MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translation Langlin Huang Mengyu Bu Yang Feng 28 0 0 03 Nov 2024
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 34 2 0 28 Oct 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 67 5 0 22 Oct 2024
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models Kushal Tatariya Vladimir Araujo Thomas Bauwens Miryam de Lhoneux VLM 33 0 0 15 Oct 2024
Wavelet-Based Image Tokenizer for Vision Transformers Zhenhai Zhu Radu Soricut ViT 42 3 0 28 May 2024
Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect Jannis Vamvas Noëmi Aepli Rico Sennrich 32 0 0 25 Jan 2024
CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code Nadezhda Chirkova Sergey Troshin 21 8 0 01 Aug 2023
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora Svanhvít Lilja Ingólfsdóttir Pétur Orri Ragnarsson H. Jónsson Haukur Barri Símonarson Vilhjálmur Þorsteinsson Vésteinn Snæbjarnarson SyDa 30 9 0 29 May 2023
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Ziwei He Meng-Da Yang Minwei Feng Jingcheng Yin X. Wang Jingwen Leng Zhouhan Lin ViT 29 11 0 24 May 2023
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models Orevaoghene Ahia Sachin Kumar Hila Gonen Jungo Kasai David R. Mortensen Noah A. Smith Yulia Tsvetkov 40 80 0 23 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 27 1 0 23 May 2023
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation Francois Meyer Jan Buys 33 8 0 11 May 2023
What is the best recipe for character-level encoder-only modelling? Kris Cao 32 2 0 09 May 2023
An Information Extraction Study: Take In Mind the Tokenization! Christos Theodoropoulos Marie-Francine Moens 24 6 0 27 Mar 2023
What do LLMs Know about Financial Markets? A Case Study on Reddit Market Sentiment Analysis Xiang Deng Vasilisa Bashlovkina Feng Han Simon Baumgartner Michael Bendersky 33 42 0 21 Dec 2022
Character-Aware Models Improve Visual Text Rendering Rosanne Liu Daniel H Garrette Chitwan Saharia William Chan Adam Roberts Sharan Narang Irina Blok R. Mical Mohammad Norouzi Noah Constant VLM 20 70 0 20 Dec 2022
ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models Jonas Belouadi Steffen Eger 46 24 0 20 Dec 2022
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training Jing-ling Huang Zhengxuan Wu Kyle Mahowald Christopher Potts 24 13 0 19 Dec 2022
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 29 25 0 13 Dec 2022
Subword-Delimited Downsampling for Better Character-Level Translation Lukas Edman Antonio Toral Gertjan van Noord 12 6 0 02 Dec 2022
Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling Zhijun Wang Xuebo Liu Min Zhang 25 11 0 23 Nov 2022
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki E. Ponti 14 42 0 17 Nov 2022
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations Borun Chen Hongyin Tang Jiahao Bu Kai Zhang Jingang Wang Qifan Wang Haitao Zheng Wei Yu Wu Liqian Yu VLM 25 1 0 23 Aug 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 30 46 0 14 Jul 2022
CONSENT: Context Sensitive Transformer for Bold Words Classification Ionut Sandu Daniel Voinea A. Popa 21 3 0 16 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Victoria Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 40 139 0 12 May 2022
UL2: Unifying Language Learning Paradigms Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason W. Wei ... Tal Schuster H. Zheng Denny Zhou N. Houlsby Donald Metzler AI4CE 57 294 0 10 May 2022
How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training? Shiyue Zhang Vishrav Chaudhary Naman Goyal James Cross Guillaume Wenzek Mohit Bansal Francisco Guzman 31 16 0 29 Apr 2022
Impact of Tokenization on Language Models: An Analysis for Turkish Cagri Toraman E. Yilmaz Furkan Şahinuç Oguzhan Ozcelik 30 74 0 19 Apr 2022
A Hierarchical N-Gram Framework for Zero-Shot Link Prediction Mingchen Li J. Chen Samuel Mensah Nikolaos Aletras Xiulong Yang Yang Ye 15 13 0 16 Apr 2022
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia Alham Fikri Aji Genta Indra Winata Fajri Koto Samuel Cahyawijaya Ade Romadhony ... David Moeljadi Radityo Eko Prasojo Timothy Baldwin Jey Han Lau Sebastian Ruder 38 98 0 24 Mar 2022
A New Generation of Perspective API: Efficient Multilingual Character-level Transformers Alyssa Lees Vinh Q. Tran Yi Tay Jeffrey Scott Sorensen Jai Gupta Donald Metzler Lucy Vasserman 25 173 0 22 Feb 2022
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP Sabrina J. Mielke Zaid Alyafeai Elizabeth Salesky Colin Raffel Manan Dey ... Arun Raja Chenglei Si Wilson Y. Lee Benoît Sagot Samson Tan 30 140 0 20 Dec 2021
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 26 15 0 26 Oct 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 32 98 0 25 Oct 2021
Why don't people use character-level machine translation? Jindrich Libovický Helmut Schmid Alexander M. Fraser 65 28 0 15 Oct 2021
Local Structure Matters Most: Perturbation Study in NLU Louis Clouâtre Prasanna Parthasarathi Amal Zouaq Sarath Chandar 22 13 0 29 Jul 2021
Evaluating Various Tokenizers for Arabic Text Classification Zaid Alyafeai Maged S. Al-Shaibani Mustafa Ghaleb Irfan Ahmad 26 41 0 14 Jun 2021
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 93 142 0 24 Oct 2020
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters Hicham El Boukkouri Olivier Ferret Thomas Lavergne Hiroshi Noji Pierre Zweigenbaum Junichi Tsujii 71 156 0 20 Oct 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 74 1,101 0 14 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 268 2,013 0 28 Jul 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 246 491 0 16 Oct 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016