Token Alignment via Character Matching for Subword Completion

13 March 2024

Papers citing "Token Alignment via Character Matching for Subword Completion"

4 / 4 papers shown

Title
Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles Buu Phan Brandon Amos Itai Gat Marton Havasi Matthew Muckley Karen Ullrich 45 1 0 11 Oct 2024
Retrieval-augmented code completion for local projects using large language models Marko Hostnik Marko Robnik-Sikonja RALM 17 0 0 09 Aug 2024
Multi-lingual Evaluation of Code Generation Models Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian ... Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth Bing Xiang ELM 109 117 0 26 Oct 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation Kaustubh D. Dhole Varun Gangal Sebastian Gehrmann Aadesh Gupta Zhenhao Li ... Tianbao Xie Usama Yaseen Michael A. Yee Jing Zhang Yue Zhang 156 86 0 06 Dec 2021