In-Context Learning with Long-Context Models: An In-Depth Exploration

30 April 2024

Graham Neubig

Papers citing "In-Context Learning with Long-Context Models: An In-Depth Exploration"

50 / 55 papers shown

Title
Bye-bye, Bluebook? Automating Legal Procedure with Large Language Models Matthew Dahl AILaw ELM 45 0 0 05 May 2025
On the generalization of language models from in-context learning and finetuning: a controlled study Andrew Kyle Lampinen Arslan Chaudhry Stephanie Chan Cody Wild Diane Wan Alex Ku Jorg Bornschein Razvan Pascanu Murray Shanahan James L. McClelland 46 0 0 01 May 2025
Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks Vishnu Sarukkai Zhiqiang Xie Kayvon Fatahalian LLMAG 68 0 0 01 May 2025
A Survey of Scaling in Large Language Model Reasoning Zihan Chen Song Wang Zhen Tan Xingbo Fu Zhenyu Lei Peng Wang Huan Liu Cong Shen Jundong Li LRM 84 0 0 02 Apr 2025
Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts Youxiang Zhu Ruochen Li Danqing Wang Daniel Haehn Xiaohui Liang LRM 53 0 0 30 Mar 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 64 0 0 17 Mar 2025
Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention Emily Xiao Chin-Jou Li Yilin Zhang Graham Neubig Amanda Bertsch BDL 63 0 0 11 Mar 2025
Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios Chao Wang Luning Zhang Z. Wang Yang Zhou ELM VLM LRM 43 1 0 27 Feb 2025
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs Jungsoo Park Junmo Kang Gabriel Stanovsky Alan Ritter 48 0 0 26 Feb 2025
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning Xuetao Ma Wenbin Jiang Hua Huang LRM 60 1 0 24 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 49 3 0 21 Feb 2025
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection Maximilian Spliethover Tim Knebler Fabian Fumagalli Maximilian Muschalik Barbara Hammer Eyke Hüllermeier Henning Wachsmuth 97 1 0 10 Feb 2025
ARISE: Iterative Rule Induction and Synthetic Data Generation for Text Classification Y. Meena Vaibhav Singh Ayush Maheshwari Amrith Krishna Ganesh Ramakrishnan AI4TS 49 0 0 09 Feb 2025
Shared DIFF Transformer Yueyang Cang Yuhang Liu Xiaoteng Zhang Xiangju Wang 31 0 0 29 Jan 2025
NExtLong: Toward Effective Long-Context Training without Long Documents Chaochen Gao Xing Wu Zijia Lin Debing Zhang Songlin Hu SyDa 64 1 0 22 Jan 2025
Better Prompt Compression Without Multi-Layer Perceptrons Edouardo Honig Andrew Lizarraga Zijun Zhang Ying Nian Wu MQ 48 0 0 12 Jan 2025
Revisiting In-Context Learning with Long Context Language Models Jinheon Baek Sun Jae Lee Prakhar Gupta Geunseob Oh Siddharth Dalmia 84 0 0 22 Dec 2024
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning Yansheng Mao Jiaqi Li Fanxu Meng Jing Xiong Zilong Zheng Muhan Zhang LLMAG RALM 90 1 0 18 Dec 2024
VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding Kangsan Kim G. Park Youngwan Lee Woongyeong Yeo Sung Ju Hwang 89 3 0 03 Dec 2024
AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations Gaurav Verma Rachneet Kaur Nishan Srishankar Zhen Zeng T. Balch Manuela Veloso LLMAG 72 5 0 20 Nov 2024
A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification Sorouralsadat Fatemi Yuheng Hu Maryam Mousavi 21 1 0 04 Nov 2024
Keep Guessing? When Considering Inference Scaling, Mind the Baselines G. Yona Or Honovich Omer Levy Roee Aharoni UQLM LRM 21 0 0 20 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 35 0 0 12 Oct 2024
Differential Transformer Tianzhu Ye Li Dong Yuqing Xia Yutao Sun Yi Zhu Gao Huang Furu Wei 35 0 0 07 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 36 3 0 06 Oct 2024
Accelerating Inference of Networks in the Frequency Domain Chenqiu Zhao Guanfang Dong Anup Basu 31 0 0 06 Oct 2024
Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization Ryan Barron Ves Grantcharov Selma Wanna Maksim E. Eren Manish Bhattarai ... George Tompkins Charles K. Nicholas Kim Ø. Rasmussen Cynthia Matuszek Boian S. Alexandrov RALM 25 3 0 03 Oct 2024
L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding? Zecheng Tang Keyan Zhou Juntao Li Baibei Ji Jianye Hou Min Zhang 31 1 0 03 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 43 24 0 03 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 55 36 0 03 Oct 2024
FlashMask: Efficient and Rich Mask Extension of FlashAttention Guoxia Wang Jinle Zeng Xiyuan Xiao Siming Wu Jiabin Yang Lujing Zheng Zeyu Chen Jiang Bian Dianhai Yu Haifeng Wang 33 2 0 02 Oct 2024
Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces Lilac Atassi 26 0 0 01 Oct 2024
A Controlled Study on Long Context Extension and Generalization in LLMs Yi Lu Jing Nathan Yan Songlin Yang Justin T. Chiu Siyu Ren Fei Yuan Wenting Zhao Zhiyong Wu Alexander M. Rush 23 9 0 18 Sep 2024
A Fused Large Language Model for Predicting Startup Success Abdurahman Maarouf Stefan Feuerriegel Nicolas Pröllochs 27 1 0 05 Sep 2024
Focused Large Language Models are Stable Many-Shot Learners Peiwen Yuan Shaoxiong Feng Yiwei Li Xinglin Wang Y. Zhang Chuyi Tan Boyuan Pan Heda Wang Yao Hu Kan Li 43 3 0 26 Aug 2024
Many-Shot In-Context Learning for Molecular Inverse Design Saeed Moayedpour Alejandro Corrochano-Navarro F. Sahneh Shahriar Noroozizadeh Alexander Koetter ... Hans Matter Christoph Grebner Gerhard Hessler Ziv Bar-Joseph Sven Jager BDL 23 2 0 26 Jul 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 23 6 0 23 Jul 2024
Private prediction for large-scale synthetic text generation Kareem Amin Alex Bie Weiwei Kong Alexey Kurakin Natalia Ponomareva Umar Syed Andreas Terzis Sergei Vassilvitskii SyDa SILM 28 2 0 16 Jul 2024
Training Task Experts through Retrieval Based Distillation Jiaxin Ge Xueying Jia Vijay Viswanathan Hongyin Luo Graham Neubig 26 2 0 07 Jul 2024
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP Omer Goldman Alon Jacovi Aviv Slobodkin Aviya Maimon Ido Dagan Reut Tsarfaty 43 10 0 29 Jun 2024
Can we teach language models to gloss endangered languages? Michael Ginn Mans Hulden Alexis Palmer 45 1 0 27 Jun 2024
Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning Brandon Huang Chancharik Mitra Assaf Arbelle Leonid Karlinsky Trevor Darrell Roei Herzig 29 10 0 21 Jun 2024
When Parts are Greater Than Sums: Individual LLM Components Can Outperform Full Models Ting-Yun Chang Jesse Thomason Robin Jia 36 4 0 19 Jun 2024
Probing the Decision Boundaries of In-context Learning in Large Language Models Siyan Zhao Tung Nguyen Aditya Grover 24 5 0 17 Jun 2024
Is In-Context Learning Sufficient for Instruction Following in LLMs? Hao Zhao Maksym Andriushchenko Francesco Croce Nicolas Flammarion 52 12 0 30 May 2024
Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words? G. Yona Roee Aharoni Mor Geva HILM 30 9 0 27 May 2024
Many-Shot In-Context Learning in Multimodal Foundation Models Yixing Jiang Jeremy Irvin Ji Hun Wang Muhammad Ahmed Chaudhry Jonathan H. Chen Andrew Y. Ng VLM 38 19 0 16 May 2024
Many-Shot In-Context Learning Rishabh Agarwal Avi Singh Lei M. Zhang Bernd Bohnet Luis Rosias ... John D. Co-Reyes Eric Chu Feryal M. P. Behbahani Aleksandra Faust Hugo Larochelle ReLM OffRL BDL 37 79 0 17 Apr 2024
In-context Learning with Retrieved Demonstrations for Language Models: A Survey an Luo Xin Xu Yue Liu Panupong Pasupat Mehran Kazemi RALM 16 54 0 21 Jan 2024
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 49 77 0 19 Sep 2023