In-Context Language Learning: Architectures and Algorithms

In-Context Language Learning: Architectures and Algorithms

23 January 2024

Bailin Wang

Papers citing "In-Context Language Learning: Architectures and Algorithms"

16 / 16 papers shown

Title
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models Tyler A. Chang Benjamin Bergen 38 0 0 21 Apr 2025
Yes, Q-learning Helps Offline In-Context RL Denis Tarasov Alexander Nikulin Ilya Zisman Albina Klepach Andrei Polubarov Nikita Lyubaykin Alexander Derevyagin Igor Kiselev Vladislav Kurenkov OffRL OnRL 80 0 0 24 Feb 2025
From Markov to Laplace: How Mamba In-Context Learns Markov Chains Marco Bondaschi Nived Rajaraman Xiuying Wei Kannan Ramchandran Razvan Pascanu Çağlar Gülçehre Michael C. Gastpar Ashok Vardhan Makkuva 58 0 0 17 Feb 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 75 4 0 31 Dec 2024
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 33 2 0 11 Nov 2024
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs Ilya Zisman Alexander Nikulin Andrei Polubarov Nikita Lyubaykin Vladislav Kurenkov Andrei Polubarov Igor Kiselev Vladislav Kurenkov OffRL 44 1 0 04 Nov 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 56 1 0 15 Jul 2024
How Effective are State Space Models for Machine Translation? Hugo Pitorro Pavlo Vasylenko Marcos Vinícius Treviso André F. T. Martins Mamba 35 2 0 07 Jul 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 64 54 0 11 Jun 2024
Memory Mosaics Jianyu Zhang Niklas Nolte Ranajoy Sadhukhan Beidi Chen Léon Bottou VLM 42 3 0 10 May 2024
Linear Transformers are Versatile In-Context Learners Max Vladymyrov J. Oswald Mark Sandler Rong Ge 21 13 0 21 Feb 2024
Zoology: Measuring and Improving Recall in Efficient Language Models Simran Arora Sabri Eyuboglu Aman Timalsina Isys Johnson Michael Poli James Zou Atri Rudra Christopher Ré 56 65 0 08 Dec 2023
Compositional Semantic Parsing with Large Language Models Andrew Drozdov Nathanael Scharli Ekin Akyuurek Nathan Scales Xinying Song Xinyun Chen Olivier Bousquet Denny Zhou ReLM LRM 187 91 0 29 Sep 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 71 220 0 21 Feb 2022
Probing Classifiers: Promises, Shortcomings, and Advances Yonatan Belinkov 221 402 0 24 Feb 2021