Investigating the Pre-Training Dynamics of In-Context Learning: Task
Recognition vs. Task Learning

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning

20 June 2024

Xiaolei Wang

Wayne Xin Zhao

Ji-Rong Wen

Papers citing "Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning"

5 / 5 papers shown

Title
ICL CIPHERS: Quantifying "Learning'' in In-Context Learning via Substitution Ciphers Zhouxiang Fang Aayush Mishra Muhan Gao Anqi Liu Daniel Khashabi 42 0 0 28 Apr 2025
DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning Xinyu Tang Xiaolei Wang Wayne Xin Zhao Ji-Rong Wen 38 3 0 26 Oct 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 130 349 0 01 Feb 2024
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 223 4,424 0 23 Jan 2020