LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

2 January 2024

Chia-Yuan Chang

Papers citing "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning"

19 / 69 papers shown

Title
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding Hanshi Sun Zhuoming Chen Xinyu Yang Yuandong Tian Beidi Chen 33 46 0 18 Apr 2024
GUARD-D-LLM: An LLM-Based Risk Assessment Engine for the Downstream uses of LLMs Sundaraparipurnan Narayanan Sandeep Vishwakarma 34 3 0 02 Apr 2024
Decoding the Digital Fine Print: Navigating the potholes in Terms of service/ use of GenAI tools against the emerging need for Transparent and Trustworthy Tech Futures Sundaraparipurnan Narayanan 16 0 0 26 Mar 2024
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding Zhenyu (Allen) Zhang Runjin Chen Shiwei Liu Zhewei Yao Olatunji Ruwase Beidi Chen Xiaoxia Wu Zhangyang Wang 21 26 0 05 Mar 2024
User-LLM: Efficient LLM Contextualization with User Embeddings Lin Ning Luyang Liu Jiaxing Wu Neo Wu D. Berlowitz Sushant Prakash Bradley Green S. O’Banion Jun Xie 42 32 0 21 Feb 2024
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration Jun Zhao Can Zu Haotian Xu Yi Lu Wei He Yiwen Ding Tao Gui Qi Zhang Xuanjing Huang RALM LLMAG 39 20 0 18 Feb 2024
LongHeads: Multi-Head Attention is Secretly a Long Context Processor Yi Lu Xin Zhou Wei He Jun Zhao Tao Ji Tao Gui Qi Zhang Xuanjing Huang LLMAG 39 11 0 16 Feb 2024
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts Kuang-Huei Lee Xinyun Chen Hiroki Furuta John F. Canny Ian S. Fischer RALM 53 29 0 15 Feb 2024
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory Chaojun Xiao Pengle Zhang Xu Han Guangxuan Xiao Yankai Lin Zhengyan Zhang Zhiyuan Liu Maosong Sun LLMAG 39 33 0 07 Feb 2024
LongAlign: A Recipe for Long Context Alignment of Large Language Models Yushi Bai Xin Lv Jiajie Zhang Yuze He Ji Qi Lei Hou Jie Tang Yuxiao Dong Juanzi Li ALM 24 44 0 31 Jan 2024
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation Zhenyu He Guhao Feng Shengjie Luo Kai-Bo Yang Liwei Wang Jingjing Xu Zhi Zhang Hongxia Yang Di He 19 13 0 29 Jan 2024
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey Yunpeng Huang Jingwei Xu Junyu Lai Zixu Jiang Taolue Chen ... Xiaoxing Ma Lijuan Yang Zhou Xin Shupeng Li Penghao Zhao LLMAG KELM 28 54 0 21 Nov 2023
Extending Input Contexts of Language Models through Training on Segmented Sequences Petros Karypis Julian McAuley George Karypis 22 0 0 23 Oct 2023
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 55 77 0 19 Sep 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond Jingfeng Yang Hongye Jin Ruixiang Tang Xiaotian Han Qizhang Feng Haoming Jiang Bing Yin Xia Hu LM&MA 123 614 0 26 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 579 0 06 Apr 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 242 695 0 27 Aug 2021
Recent Advances in Adversarial Training for Adversarial Robustness Tao Bai Jinqi Luo Jun Zhao B. Wen Qian Wang AAML 71 473 0 02 Feb 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 2,009 0 28 Jul 2020