ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2303.07971
  4. Cited By
A Theory of Emergent In-Context Learning as Implicit Structure Induction

A Theory of Emergent In-Context Learning as Implicit Structure Induction

14 March 2023
Michael Hahn
Navin Goyal
    LRM
ArXiv (abs)PDFHTMLGithub

Papers citing "A Theory of Emergent In-Context Learning as Implicit Structure Induction"

50 / 66 papers shown
Genomic Next-Token Predictors are In-Context Learners
Genomic Next-Token Predictors are In-Context Learners
Nathan Breslow
Aayush Mishra
Mahler Revsine
Michael C. Schatz
Anqi Liu
Daniel Khashabi
275
0
0
16 Nov 2025
Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training
Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training
Dake Bu
Wei Huang
Andi Han
Atsushi Nitanda
Hau-San Wong
Qingfu Zhang
Taiji Suzuki
LRM
267
1
0
10 Nov 2025
Hyperspectral data augmentation with transformer-based diffusion models
Hyperspectral data augmentation with transformer-based diffusion models
Mattia Ferrari
Lorenzo Bruzzone
155
1
0
09 Oct 2025
Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning
Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning
Tianle Zhang
Wanlong Fang
Jonathan Woo
Paridhi Latawa
Deepak A.Subramanian
Alvin Chan
267
2
0
22 Sep 2025
InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversity
InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversity
Souradeep Nanda
Anay Majee
Rishabh K. Iyer
175
0
0
28 Aug 2025
The Other Mind: How Language Models Exhibit Human Temporal Cognition
The Other Mind: How Language Models Exhibit Human Temporal Cognition
Lingyu Li
Yang Yao
Yixu Wang
Chubo Li
Yan Teng
Y. Wang
263
3
0
21 Jul 2025
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Léo Gagnon
Eric Elmoznino
Sarthak Mittal
Tom Marty
Tejas Kasetty
Dhanya Sridhar
Guillaume Lajoie
265
0
0
19 Jun 2025
Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
Chengye Li
Haiyun Liu
Yuanxi Li
279
2
0
13 Jun 2025
Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
Harish Tayyar Madabushi
Melissa Torgbi
C. Bonial
467
4
0
29 May 2025
Mechanistic evaluation of Transformers and state space models
Mechanistic evaluation of Transformers and state space models
Aryaman Arora
Neil Rathi
Nikil Roashan Selvam
Róbert Csordás
Dan Jurafsky
Christopher Potts
505
4
0
21 May 2025
ICL CIPHERS: Quantifying "Learning" in In-Context Learning via Substitution Ciphers
ICL CIPHERS: Quantifying "Learning" in In-Context Learning via Substitution Ciphers
Zhouxiang Fang
Aayush Mishra
Muhan Gao
Anqi Liu
Daniel Khashabi
535
3
0
28 Apr 2025
When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
Rei Higuchi
Ryotaro Kawata
Naoki Nishikawa
Kazusato Oko
Shoichiro Yamaguchi
Sosuke Kobayashi
Seiya Tokui
K. Hayashi
Daisuke Okanohara
Taiji Suzuki
AI4CE
426
2
0
24 Apr 2025
Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B
Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B
Aleksandra Bakalova
Yana Veitsman
Xinting Huang
Michael Hahn
382
9
0
31 Mar 2025
Enough Coin Flips Can Make LLMs Act Bayesian
Enough Coin Flips Can Make LLMs Act BayesianAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Ritwik Gupta
Rodolfo Corona
Jiaxin Ge
Eric Wang
Dan Klein
Trevor Darrell
David M. Chan
BDLLRM
383
17
0
06 Mar 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers
Alireza Amiri
Xinting Huang
Mark Rofin
Michael Hahn
LRM
1.4K
24
0
04 Feb 2025
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?International Conference on Learning Representations (ICLR), 2025
Yutong Yin
Zhaoran Wang
LRMReLM
1.4K
3
0
27 Jan 2025
Using Pre-trained LLMs for Multivariate Time Series Forecasting
Using Pre-trained LLMs for Multivariate Time Series Forecasting
Malcolm Wolff
Shenghao Yang
Kari Torkkola
Michael W. Mahoney
AI4TSAIFin
293
6
0
10 Jan 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers
Out-of-distribution generalization via composition: a lens through induction heads in TransformersProceedings of the National Academy of Sciences of the United States of America (PNAS), 2024
Jiajun Song
Zhuoyan Xu
Yiqiao Zhong
405
26
0
31 Dec 2024
Conceptual In-Context Learning and Chain of Concepts: Solving Complex
  Conceptual Problems Using Large Language Models
Conceptual In-Context Learning and Chain of Concepts: Solving Complex Conceptual Problems Using Large Language Models
Nishtha N. Vaidya
Thomas Runkler
Thomas Hubauer
Veronika Haderlein-Hoegberg
Maja Mlicic Brandt
LRM
299
1
0
19 Dec 2024
Bayesian scaling laws for in-context learning
Bayesian scaling laws for in-context learning
Aryaman Arora
Dan Jurafsky
Christopher Potts
Noah D. Goodman
663
15
0
21 Oct 2024
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures
Wei Shen
Ruida Zhou
Jing Yang
Cong Shen
533
12
0
15 Oct 2024
Racing Thoughts: Explaining Contextualization Errors in Large Language Models
Racing Thoughts: Explaining Contextualization Errors in Large Language ModelsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024
Michael A. Lepori
Michael Mozer
Asma Ghandeharioun
LRM
597
1
0
02 Oct 2024
In-Context Learning with Representations: Contextual Generalization of
  Trained Transformers
In-Context Learning with Representations: Contextual Generalization of Trained TransformersNeural Information Processing Systems (NeurIPS), 2024
Tong Yang
Yu Huang
Yingbin Liang
Yuejie Chi
MLT
374
35
0
19 Aug 2024
Representing Rule-based Chatbots with Transformers
Representing Rule-based Chatbots with Transformers
Dan Friedman
Abhishek Panigrahi
Danqi Chen
476
2
0
15 Jul 2024
Estimating the Hallucination Rate of Generative AI
Estimating the Hallucination Rate of Generative AI
Andrew Jesson
Nicolas Beltran-Velez
Quentin Chu
Sweta Karlekar
Jannik Kossen
Yarin Gal
John P. Cunningham
David M. Blei
594
35
0
11 Jun 2024
On Subjective Uncertainty Quantification and Calibration in Natural
  Language Generation
On Subjective Uncertainty Quantification and Calibration in Natural Language GenerationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024
Ziyu Wang
Chris Holmes
UQLM
545
14
0
07 Jun 2024
What Do Language Models Learn in Context? The Structured Task Hypothesis
What Do Language Models Learn in Context? The Structured Task HypothesisAnnual Meeting of the Association for Computational Linguistics (ACL), 2024
Jiaoda Li
Buse Giledereli
Mrinmaya Sachan
Robert Bamler
LRM
489
17
0
06 Jun 2024
Is In-Context Learning in Large Language Models Bayesian? A Martingale
  Perspective
Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective
Fabian Falck
Ziyu Wang
Chris Holmes
450
44
0
02 Jun 2024
How In-Context Learning Emerges from Training on Unstructured Data: On
  the Role of Co-Occurrence, Positional Information, and Noise Structures
How In-Context Learning Emerges from Training on Unstructured Data: On the Role of Co-Occurrence, Positional Information, and Noise Structures
Kevin Christian Wibisono
Yixin Wang
161
0
0
31 May 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of
  LLM-Driven Autonomous Systems
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
Jianliang He
Siyu Chen
Fengzhuo Zhang
Zhuoran Yang
LM&RoLLMAG
357
12
0
30 May 2024
Does learning the right latent variables necessarily improve in-context learning?
Does learning the right latent variables necessarily improve in-context learning?
Sarthak Mittal
Eric Elmoznino
Léo Gagnon
Sangnie Bhardwaj
Tom Marty
Dhanya Sridhar
Guillaume Lajoie
477
9
0
29 May 2024
Finding Visual Task Vectors
Finding Visual Task Vectors
Alberto Hojel
Yutong Bai
Trevor Darrell
Amir Globerson
Amir Bar
332
20
0
08 Apr 2024
Can large language models explore in-context?
Can large language models explore in-context?Neural Information Processing Systems (NeurIPS), 2024
Akshay Krishnamurthy
Keegan Harris
Dylan J. Foster
Cyril Zhang
Aleksandrs Slivkins
LM&RoLLMAGLRM
717
63
0
22 Mar 2024
Concept-aware Data Construction Improves In-context Learning of Language
  Models
Concept-aware Data Construction Improves In-context Learning of Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024
Michal Štefánik
Marek Kadlcík
Petr Sojka
325
2
0
08 Mar 2024
LLM Task Interference: An Initial Study on the Impact of Task-Switch in
  Conversational History
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History
Akash Gupta
Ivaxi Sheth
Vyas Raina
Mark Gales
Mario Fritz
408
20
0
28 Feb 2024
Visual In-Context Learning for Large Vision-Language Models
Visual In-Context Learning for Large Vision-Language Models
Yucheng Zhou
Xiang Li
Qianning Wang
Jianbing Shen
MLLM
276
134
0
18 Feb 2024
Pelican Soup Framework: A Theoretical Framework for Language Model Capabilities
Pelican Soup Framework: A Theoretical Framework for Language Model Capabilities
Ting-Rui Chiang
Dani Yogatama
266
4
0
16 Feb 2024
Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning
  Tasks
Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park
Jaeseung Park
Zheyang Xiong
Nayoung Lee
Jaewoong Cho
Samet Oymak
Kangwook Lee
Dimitris Papailiopoulos
463
121
0
06 Feb 2024
Learning Universal Predictors
Learning Universal PredictorsInternational Conference on Machine Learning (ICML), 2024
Jordi Grau-Moya
Tim Genewein
Marcus Hutter
Laurent Orseau
Grégoire Delétang
...
Anian Ruoss
Wenliang Kevin Li
Christopher Mattern
Matthew Aitchison
J. Veness
288
29
0
26 Jan 2024
Demystifying Chains, Trees, and Graphs of Thoughts
Demystifying Chains, Trees, and Graphs of ThoughtsIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024
Maciej Besta
Florim Memedi
Zhenyu Zhang
Robert Gerstenberger
Guangyuan Piao
...
Aleš Kubíček
H. Niewiadomski
Aidan O'Mahony
Onur Mutlu
Torsten Hoefler
AI4CELRM
1.2K
71
0
25 Jan 2024
In-Context Language Learning: Architectures and Algorithms
In-Context Language Learning: Architectures and AlgorithmsInternational Conference on Machine Learning (ICML), 2024
Ekin Akyürek
Bailin Wang
Yoon Kim
Jacob Andreas
LRMReLM
451
90
0
23 Jan 2024
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for
  In-context Learning
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context LearningConference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Shuai Zhao
Meihuizi Jia
Anh Tuan Luu
Fengjun Pan
Jinming Wen
AAML
591
77
0
11 Jan 2024
Generalization to New Sequential Decision Making Tasks with In-Context
  Learning
Generalization to New Sequential Decision Making Tasks with In-Context Learning
Sharath Chandra Raparthy
Eric Hambro
Robert Kirk
Mikael Henaff
Roberta Raileanu
OffRL
387
37
0
06 Dec 2023
How are Prompts Different in Terms of Sensitivity?
How are Prompts Different in Terms of Sensitivity?North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Sheng Lu
Hendrik Schuff
Iryna Gurevych
394
35
0
13 Nov 2023
Gen-Z: Generative Zero-Shot Text Classification with Contextualized
  Label Descriptions
Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label DescriptionsInternational Conference on Learning Representations (ICLR), 2023
Sachin Kumar
Chan Young Park
Yulia Tsvetkov
VLM
286
8
0
13 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on
  Interpretation and Analysis
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and AnalysisConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Yuxiang Zhou
Jiazheng Li
Yanzheng Xiang
Hanqi Yan
Lin Gui
Yulan He
390
43
0
01 Nov 2023
Which Examples to Annotate for In-Context Learning? Towards Effective
  and Efficient Selection
Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection
Costas Mavromatis
Ninad Kulkarni
Zhengyuan Shen
Jiani Zhang
Huzefa Rangwala
Christos Faloutsos
George Karypis
295
42
0
30 Oct 2023
In-Context Learning Dynamics with Random Binary Sequences
In-Context Learning Dynamics with Random Binary SequencesInternational Conference on Learning Representations (ICLR), 2023
Eric J. Bigelow
Ekdeep Singh Lubana
Robert P. Dick
Hidenori Tanaka
T. Ullman
518
12
0
26 Oct 2023
Function Vectors in Large Language Models
Function Vectors in Large Language ModelsInternational Conference on Learning Representations (ICLR), 2023
Eric Todd
Millicent Li
Arnab Sen Sharma
Aaron Mueller
Byron C. Wallace
David Bau
485
217
0
23 Oct 2023
Do pretrained Transformers Learn In-Context by Gradient Descent?
Do pretrained Transformers Learn In-Context by Gradient Descent?
Lingfeng Shen
Aayush Mishra
Daniel Khashabi
481
14
0
12 Oct 2023
12
Next
Page 1 of 2