When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks

15 November 2023

Hao Peng

Xiaozhi Wang

Bin Xu

Lei Hou

Juanzi Li

ArXiv PDF HTML

Papers citing "When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks"

32 / 32 papers shown

Title
LLMs Get Lost In Multi-Turn Conversation Philippe Laban Hiroaki Hayashi Yingbo Zhou Jennifer Neville 25 0 0 09 May 2025
Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study Faeze Ghorbanpour Daryna Dementieva Alexander M. Fraser 23 0 0 09 May 2025
Scaling sparse feature circuit finding for in-context learning Dmitrii Kharlapenko S. Kamath S Fazl Barez Arthur Conmy Neel Nanda 23 0 0 18 Apr 2025
Grounding Agent Reasoning in Image Schemas: A Neurosymbolic Approach to Embodied Cognition François Olivier Zied Bouraoui LM&Ro 36 0 0 31 Mar 2025
Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection Jianfei He Lilin Wang Jiaying Wang Zhenyu Liu Hongbin Na Z. Wang Wei Wang Qi Chen 18 2 0 21 Oct 2024
In-Context Learning for Long-Context Sentiment Analysis on Infrastructure Project Opinions Alireza Shamshiri Kyeong Rok Ryu June Young Park LLMAG 14 1 0 15 Oct 2024
Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering Zimu Wang Lei Xia Wei Wang Xinya Du 13 4 0 07 Oct 2024
InfiniPot: Infinite Context Processing on Memory-Constrained LLMs Minsoo Kim Kyuhong Shim Jungwook Choi Simyung Chang 13 5 0 02 Oct 2024
Unsupervised Human Preference Learning Sumuk Shashidhar Abhinav Chinta Vaibhav Sahai Dilek Hakkani Tur LRM 25 0 0 30 Sep 2024
MAVEN-Fact: A Large-scale Event Factuality Detection Dataset Chunyang Li Hao Peng Xiaozhi Wang Y. Qi Lei Hou Bin Xu Juanzi Li HILM 25 1 0 22 Jul 2024
LLMAEL: Large Language Models are Good Context Augmenters for Entity Linking Amy Xin Y. Qi Zijun Yao Fangwei Zhu Kaisheng Zeng Xu Bin Lei Hou Juanzi Li 21 5 0 04 Jul 2024
ADELIE: Aligning Large Language Models on Information Extraction Y. Qi Hao Peng Xiaozhi Wang Bin Xu Lei Hou Juanzi Li 19 7 0 08 May 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph Xiaochen Kev Gao Feng Yao Kewen Zhao Beilei He Animesh Kumar Vish Krishnan Jingbo Shang 22 0 0 22 Apr 2024
Long-context LLMs Struggle with Long In-context Learning Tianle Li Ge Zhang Quy Duc Do Xiang Yue Wenhu Chen 36 152 0 02 Apr 2024
Rethinking Human-like Translation Strategy: Integrating Drift-Diffusion Model with Large Language Models for Machine Translation Hongbin Na Zimu Wang M. Maimaiti Tong Chen Wei Wang Tao Shen Ling Chen LRM 13 5 0 16 Feb 2024
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction Frank Mtumbuka Steven Schockaert 15 0 0 18 Dec 2023
Generating Valid and Natural Adversarial Examples with Large Language Models Zimu Wang Wei Wang Qi Chen Qiufeng Wang Anh Nguyen AAML 8 4 0 20 Nov 2023
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation Xiaozhi Wang Hao Peng Yong Guan Kaisheng Zeng Jianhui Chen ... Yankai Lin Zhiyuan Liu Ruobing Xie Jie Zhou Juanzi Li 8 4 0 15 Nov 2023
Position: AI Evaluation Should Learn from How We Test Humans Yan Zhuang Q. Liu Yuting Ning Wei Huang Rui Lv Zhenya Huang Guanhao Zhao Zheng-Wei Zhang ELM ALM 62 21 0 18 Jun 2023
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts Jian Xie Kai Zhang Jiangjie Chen Renze Lou Yu-Chuan Su RALM 198 150 0 22 May 2023
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction? Xin Xu Yuqi Zhu Xiaohan Wang Ningyu Zhang KELM LRM 43 45 0 02 May 2023
On the Creativity of Large Language Models Giorgio Franceschelli Mirco Musolesi 54 48 0 27 Mar 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 216 327 0 23 Aug 2022
DeepStruct: Pretraining of Language Models for Structure Prediction Chenguang Wang Xiao Liu Zui Chen Haoyun Hong Jie Tang Dawn Song 209 66 0 21 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 233 236 0 10 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 274 882 0 18 Apr 2021
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models Alex Tamkin Miles Brundage Jack Clark Deep Ganguli AILaw ELM 192 248 0 04 Feb 2021
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 227 1,281 0 18 Mar 2020
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification Tianyu Gao Xu Han Hao Zhu Zhiyuan Liu Peng Li Maosong Sun Jie Zhou 195 221 0 16 Oct 2019