Taking a Deep Breath: Enhancing Language Modeling of Large Language
Models with Sentinel Tokens

Taking a Deep Breath: Enhancing Language Modeling of Large Language Models with Sentinel Tokens

16 June 2024

Heming Xia

Weikang Wang

Tianyu Liu

Zhifang Sui

Papers citing "Taking a Deep Breath: Enhancing Language Modeling of Large Language Models with Sentinel Tokens"

4 / 4 papers shown

Title
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 122 1 0 03 Apr 2025
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 251 2,012 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 244 491 0 16 Oct 2019