Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering

20 December 2022

Zhiyong Wu

Yaoxiang Wang

Jiacheng Ye

Lingpeng Kong

ArXiv PDF HTML

Papers citing "Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering"

47 / 97 papers shown

Title
Interpretable Cross-Examination Technique (ICE-T): Using highly informative features to boost LLM performance Goran Muric Ben Delay Steven Minton 24 1 0 08 May 2024
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? Yang Luo Zangwei Zheng Zirui Zhu Yang You 41 5 0 19 Apr 2024
Long-context LLMs Struggle with Long In-context Learning Tianle Li Ge Zhang Quy Duc Do Xiang Yue Wenhu Chen 45 155 0 02 Apr 2024
Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction Chenming Tang Fanyi Qu Yunfang Wu 31 1 0 28 Mar 2024
TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision Ruiwen Zhou Yingxuan Yang Kangrui Chen Ying Wen Wenhao Wang Chunling Xi Guoqiang Xu Jiliang Tang Lingjuan Lyu LLMAG 24 8 0 10 Mar 2024
In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Model Junhui Yin Xinyu Zhang Lin Wu Xianghua Xie Xiaojie Wang VPVLM VLM MLLM 25 2 0 10 Mar 2024
Say More with Less: Understanding Prompt Learning Behaviors through Gist Compression Xinze Li Zhenghao Liu Chenyan Xiong Shi Yu Yukun Yan Shuo Wang Ge Yu VLM 36 4 0 25 Feb 2024
Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 29 6 0 23 Feb 2024
How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? Hongkang Li Meng Wang Songtao Lu Xiaodong Cui Pin-Yu Chen MLT 32 14 0 23 Feb 2024
$Se^2$ : Sequential Example Selection for In-Context Learning Haoyu Liu Jianfeng Liu Shaohan Huang Yuefeng Zhan Hao Sun Weiwei Deng Furu Wei Qi Zhang 25 3 0 21 Feb 2024
In-Context Learning Demonstration Selection via Influence Analysis Vinay M.S. Minh-Hao Van Xintao Wu 29 3 0 19 Feb 2024
In-Context Example Ordering Guided by Label Distributions Zhichao Xu Daniel Cohen Bei Wang Vivek Srikumar 22 7 0 18 Feb 2024
Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning Yinpeng Liu Jiawei Liu Xiang Shi Qikai Cheng Yong Huang Wei Lu 23 24 0 16 Feb 2024
Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation Jiashu Pu Yajing Wan Yuru Zhang Jing Chen Ling Cheng Qian Shao Yongzhu Chang Tangjie Lv Rongsheng Zhang 21 1 0 15 Feb 2024
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models Jiacheng Ye Shansan Gong Liheng Chen Lin Zheng Jiahui Gao ... Chuan Wu Xin Jiang Zhenguo Li Wei Bi Lingpeng Kong DiffM LRM AI4CE 38 12 0 12 Feb 2024
Automatic Combination of Sample Selection Strategies for Few-Shot Learning Branislav Pecher Ivan Srba M. Bieliková Joaquin Vanschoren 26 1 0 05 Feb 2024
Revisiting Demonstration Selection Strategies in In-Context Learning Keqin Peng Liang Ding Yancheng Yuan Xuebo Liu Min Zhang Y. Ouyang Dacheng Tao 25 20 0 22 Jan 2024
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 29 25 0 15 Jan 2024
Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements Anton Voronov Lena Wolf Max Ryabinin 22 46 0 12 Jan 2024
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning Yiqi Wang Wentao Chen Xiaotian Han Xudong Lin Haiteng Zhao Yongfei Liu Bohan Zhai Jianbo Yuan Quanzeng You Hongxia Yang LRM 33 66 0 10 Jan 2024
Grimoire is All You Need for Enhancing Large Language Models Ding Chen Shichao Song Qingchen Yu Zhiyu Li Wenjin Wang Feiyu Xiong Bo Tang 25 4 0 07 Jan 2024
Supervised Knowledge Makes Large Language Models Better In-context Learners Linyi Yang Shuibai Zhang Zhuohao Yu Guangsheng Bao Yidong Wang ... Ruochen Xu Weirong Ye Xing Xie Weizhu Chen Yue Zhang 16 14 0 26 Dec 2023
Think from Words(TFW): Initiating Human-Like Cognition in Large Language Models Through Think from Words for Japanese Text-level Classification Chengguang Gan Qinghao Zhang Tatsunori Mori 19 1 0 06 Dec 2023
GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks Shivanshu Gupta Clemens Rosenbaum Ethan R. Elenberg LRM 29 6 0 16 Nov 2023
Instruct Me More! Random Prompting for Visual In-Context Learning Jiahao Zhang Bowen Wang Liangzhi Li Yuta Nakashima Hajime Nagahara VLM 20 15 0 07 Nov 2023
Retrieval-Augmented Chain-of-Thought in Semi-structured Domains Vaibhav Mavi Abulhair Saparov Chen Zhao LRM 27 4 0 22 Oct 2023
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning Hongfu Liu Ye Wang 29 9 0 13 Oct 2023
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang Lili Qiu RALM 101 182 0 10 Oct 2023
EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling Siyu Ren Zhiyong Wu Kenny Q. Zhu 18 3 0 07 Oct 2023
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models Wei Qi Leong Jian Gang Ngui Yosephine Susanto Hamsawardhini Rengarajan Kengatharaiyer Sarveswaran William-Chandra Tjhi 21 9 0 12 Sep 2023
Approximating Human-Like Few-shot Learning with GPT-based Compression C.-Y. Huang Yuqing Xie Zhiying Jiang Jimmy J. Lin Ming Li 9 9 0 14 Aug 2023
GPT-FinRE: In-context Learning for Financial Relation Extraction using Large Language Models P. Rajpoot Ankur P. Parikh 16 14 0 30 Jun 2023
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting J. Wang Qiushi Sun Xiang Li Ming Gao ReLM LRM 19 64 0 10 Jun 2023
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models Yew Ken Chia Pengfei Hong Lidong Bing Soujanya Poria ELM 25 61 0 07 Jun 2023
Coverage-based Example Selection for In-Context Learning Shivanshu Gupta Matt Gardner Sameer Singh 18 39 0 24 May 2023
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis Sohee Yang Jonghyeon Kim Joel Jang Seonghyeon Ye Hyunji Lee Minjoon Seo 28 9 0 24 May 2023
Skill-Based Few-Shot Selection for In-Context Learning Shengnan An Bo Zhou Zeqi Lin Qiang Fu B. Chen Nanning Zheng Weizhu Chen Jian-Guang Lou 21 31 0 23 May 2023
Generating Data for Symbolic Language with Large Language Models Jiacheng Ye Chengzu Li Lingpeng Kong Tao Yu 28 10 0 23 May 2023
Iterative Forward Tuning Boosts In-Context Learning in Language Models Jiaxi Yang Binyuan Hui Min Yang Bailin Wang Bowen Li Binhua Li Fei Huang Yongbin Li 25 16 0 22 May 2023
Large Language Model Instruction Following: A Survey of Progresses and Challenges Renze Lou Kai Zhang Wenpeng Yin ALM LRM 27 19 0 18 Mar 2023
In-Context Learning with Many Demonstration Examples Mukai Li Shansan Gong Jiangtao Feng Yiheng Xu Jinchao Zhang Zhiyong Wu Lingpeng Kong 32 32 0 09 Feb 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 20 460 0 31 Dec 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 277 1,114 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,311 0 17 Jan 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 245 671 0 06 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,913 0 31 Dec 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018