ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2112.04426
  4. Cited By
Improving language models by retrieving from trillions of tokens
v1v2v3 (latest)

Improving language models by retrieving from trillions of tokens

8 December 2021
Sebastian Borgeaud
A. Mensch
Jordan Hoffmann
Trevor Cai
Eliza Rutherford
Katie Millican
George van den Driessche
Jean-Baptiste Lespiau
Bogdan Damoc
Aidan Clark
Diego de Las Casas
Aurelia Guy
Jacob Menick
Roman Ring
Tom Hennigan
Saffron Huang
Lorenzo Maggiore
Chris Jones
Albin Cassirer
Andy Brock
Michela Paganini
G. Irving
Oriol Vinyals
Simon Osindero
Karen Simonyan
Jack W. Rae
Erich Elsen
Laurent Sifre
    KELMRALM
ArXiv (abs)PDFHTMLHuggingFace (1 upvotes)

Papers citing "Improving language models by retrieving from trillions of tokens"

50 / 893 papers shown
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to
  the Edge of Generalization
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
Boshi Wang
Xiang Yue
Yu-Chuan Su
Huan Sun
LRM
379
74
0
23 May 2024
Automated Evaluation of Retrieval-Augmented Language Models with
  Task-Specific Exam Generation
Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation
Gauthier Guinet
Behrooz Omidvar-Tehrani
Hao Ding
Laurent Callot
RALM
279
33
0
22 May 2024
FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
Jiajie Jin
Yutao Zhu
Xinyu Yang
Chenghao Zhang
Zhicheng Dou
Chenghao Zhang
Tong Zhao
Zhao Yang
Zhicheng Dou
Ji-Rong Wen
VLM
414
139
0
22 May 2024
Towards Retrieval-Augmented Architectures for Image Captioning
Towards Retrieval-Augmented Architectures for Image Captioning
Sara Sarto
Marcella Cornia
Lorenzo Baraldi
Alessandro Nicolosi
Rita Cucchiara
VLM
241
18
0
21 May 2024
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
William Brandon
Mayank Mishra
Aniruddha Nrusimha
Yikang Shen
Jonathan Ragan-Kelley
MQ
257
88
0
21 May 2024
Information Leakage from Embedding in Large Language Models
Information Leakage from Embedding in Large Language Models
Zhipeng Wan
Anda Cheng
Yinggui Wang
Lei Wang
PILM
250
7
0
20 May 2024
PyZoBot: A Platform for Conversational Information Extraction and
  Synthesis from Curated Zotero Reference Libraries through Advanced
  Retrieval-Augmented Generation
PyZoBot: A Platform for Conversational Information Extraction and Synthesis from Curated Zotero Reference Libraries through Advanced Retrieval-Augmented Generation
S. Alshammari
Lama Basalelah
Walaa Abu Rukbah
Ali Alsuhibani
D. Wijesinghe
128
0
0
13 May 2024
DuetRAG: Collaborative Retrieval-Augmented Generation
DuetRAG: Collaborative Retrieval-Augmented Generation
Dian Jiao
Li Cai
Jingsheng Huang
Wenqiao Zhang
Siliang Tang
Yueting Zhuang
168
1
0
12 May 2024
Large Language Models for Education: A Survey
Large Language Models for Education: A Survey
Hanyi Xu
Wensheng Gan
Zhenlian Qi
Jiayang Wu
Philip S. Yu
AI4EdELM
320
52
0
12 May 2024
AIOS Compiler: LLM as Interpreter for Natural Language Programming and
  Flow Programming of AI Agents
AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents
Shuyuan Xu
Zelong Li
Kai Mei
Zelong Li
190
10
0
11 May 2024
Redefining Information Retrieval of Structured Database via Large
  Language Models
Redefining Information Retrieval of Structured Database via Large Language Models
Mingzhu Wang
Yuzhe Zhang
Qihang Zhao
Juanyi Yang
Kuanqi Cai
RALMKELM
212
2
0
09 May 2024
FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference
FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference
Runheng Liu
Xingchen Xiao
Heyan Huang
Zewen Chi
Zhijing Wu
RALMKELM
346
1
0
07 May 2024
BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine
BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine
Mingchen Li
H. Kilicoglu
Hualei Xu
Rui Zhang
LM&MARALM
406
55
0
01 May 2024
When to Retrieve: Teaching LLMs to Utilize Information Retrieval
  Effectively
When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively
Tiziano Labruna
Jon Ander Campos
Gorka Azkune
216
19
0
30 Apr 2024
HELPER-X: A Unified Instructable Embodied Agent to Tackle Four
  Interactive Vision-Language Domains with Memory-Augmented Language Models
HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models
Gabriel H. Sarch
Sahil Somani
Raghav Kapoor
Michael J. Tarr
Katerina Fragkiadaki
LM&RoLLMAG
269
6
0
29 Apr 2024
From Persona to Personalization: A Survey on Role-Playing Language
  Agents
From Persona to Personalization: A Survey on Role-Playing Language Agents
Jiangjie Chen
Xintao Wang
Rui Xu
Siyu Yuan
Yikai Zhang
...
Caiyu Hu
Siye Wu
Scott Ren
Ziquan Fu
Yanghua Xiao
384
181
0
28 Apr 2024
Studying Large Language Model Behaviors Under Realistic Knowledge
  Conflicts
Studying Large Language Model Behaviors Under Realistic Knowledge Conflicts
Evgenii Kortukov
Alexander Rubinstein
Elisa Nguyen
Seong Joon Oh
RALM
1.2K
5
2
24 Apr 2024
Graph Machine Learning in the Era of Large Language Models (LLMs)
Graph Machine Learning in the Era of Large Language Models (LLMs)
Wenqi Fan
Shijie Wang
Jiani Huang
Zhikai Chen
Yu Song
...
Haitao Mao
Hui Liu
Xiaorui Liu
D. Yin
Qing Li
AI4CE
429
43
0
23 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval
From Matching to Generation: A Survey on Generative Information Retrieval
Xiaoxi Li
Jiajie Jin
Yujia Zhou
Yuyao Zhang
Peitian Zhang
Yutao Zhu
Zhicheng Dou
3DV
551
132
0
23 Apr 2024
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
Chao Jin
Zili Zhang
Xuanlin Jiang
Fangyue Liu
Xin Liu
Xuanzhe Liu
Xin Jin
377
79
0
18 Apr 2024
A Survey on Retrieval-Augmented Text Generation for Large Language
  Models
A Survey on Retrieval-Augmented Text Generation for Large Language Models
Yizheng Huang
Jimmy X. Huang
3DVRALM
318
91
0
17 Apr 2024
SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA
  of LLMs
SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs
Jaehyung Kim
Jaehyun Nam
Sangwoo Mo
Jongjin Park
Sang-Woo Lee
Minjoon Seo
Jung-Woo Ha
Jinwoo Shin
AIFinRALMELM
322
83
0
17 Apr 2024
Vocabulary-free Image Classification and Semantic Segmentation
Vocabulary-free Image Classification and Semantic Segmentation
Alessandro Conti
Enrico Fini
Goran Frehse
Paolo Rota
Yiming Wang
Elisa Ricci
VLM
221
7
0
16 Apr 2024
Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large
  Language Model for Domain Question Answering
Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering
Yuqi Wang
Boran Jiang
Yi Luo
Dawei He
Peng Cheng
Liangcai Gao
LRM
93
5
0
16 Apr 2024
Compression Represents Intelligence Linearly
Compression Represents Intelligence Linearly
Yuzhen Huang
Jinghan Zhang
Zifei Shan
Junxian He
216
39
0
15 Apr 2024
kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually
  Expanding Large Vocabularies
kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabularies
Zhongrui Gui
Shuyang Sun
Runjia Li
Jianhao Yuan
Zhaochong An
Karsten Roth
Christian Schroeder de Witt
Juil Sock
VLMCLL
285
18
0
15 Apr 2024
Best Practices and Lessons Learned on Synthetic Data for Language Models
Best Practices and Lessons Learned on Synthetic Data for Language Models
Ruibo Liu
Jerry W. Wei
Fangyu Liu
Chenglei Si
Yanzhe Zhang
...
Steven Zheng
Daiyi Peng
Diyi Yang
Denny Zhou
Andrew M. Dai
SyDaEgoV
304
112
0
11 Apr 2024
Superposition Prompting: Improving and Accelerating Retrieval-Augmented
  Generation
Superposition Prompting: Improving and Accelerating Retrieval-Augmented GenerationInternational Conference on Machine Learning (ICML), 2024
Thomas Merth
Qichen Fu
Mohammad Rastegari
Mahyar Najibi
LRMRALM
358
13
0
10 Apr 2024
Privacy Preserving Prompt Engineering: A Survey
Privacy Preserving Prompt Engineering: A Survey
Kennedy Edemacu
Xintao Wu
380
37
0
09 Apr 2024
RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with
  Multimodal Large Language Models
RoboMP2^22: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models
Qi Lv
Haochuan Li
Xiang Deng
Rui Shao
Michael Yu Wang
Liqiang Nie
LRMLM&Ro
231
4
0
07 Apr 2024
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
Jingyu Zhang
Marc Marone
Tianjian Li
Benjamin Van Durme
Daniel Khashabi
582
13
0
05 Apr 2024
How Easily do Irrelevant Inputs Skew the Responses of Large Language
  Models?
How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?
Siye Wu
Jian Xie
Jiangjie Chen
Tinghui Zhu
Kai Zhang
Yanghua Xiao
KELM
299
34
0
04 Apr 2024
Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing
  Positional Bias in LLMs
Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs
Zheng Zhang
Fan Yang
Ziyan Jiang
Zheng Chen
Zhengyang Zhao
Chengyuan Ma
Bo Pan
Yang Liu
97
9
0
01 Apr 2024
Source-Aware Training Enables Knowledge Attribution in Language Models
Source-Aware Training Enables Knowledge Attribution in Language Models
Muhammad Khalifa
Aman Rangapur
Emma Strubell
Honglak Lee
Lu Wang
Iz Beltagy
Hao Peng
HILM
403
26
0
01 Apr 2024
SOAR: Improved Indexing for Approximate Nearest Neighbor Search
SOAR: Improved Indexing for Approximate Nearest Neighbor Search
Philip Sun
David Simcha
Dave Dopson
Ruiqi Guo
Sanjiv Kumar
209
17
0
31 Mar 2024
Towards a Robust Retrieval-Based Summarization System
Towards a Robust Retrieval-Based Summarization System
Shengjie Liu
Jing Wu
Jingyuan Bao
Wenyi Wang
N. Hovakimyan
Christopher G. Healey
RALM
184
13
0
29 Mar 2024
Quantum Natural Language Processing
Quantum Natural Language Processing
Dominic Widdows
Willie Aboumrad
Dohun Kim
Sayonee Ray
Jonathan Mei
303
8
0
28 Mar 2024
RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in
  Instructional Videos
RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in Instructional Videos
Ali Zare
Yulei Niu
Hammad A. Ayyubi
Shih-Fu Chang
219
4
0
27 Mar 2024
BLADE: Enhancing Black-box Large Language Models with Small
  Domain-Specific Models
BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
Haitao Li
Jiaxin Mao
Jia Chen
Qian Dong
Zhijing Wu
Yiqun Liu
Chong Chen
Qi Tian
AILaw
227
24
0
27 Mar 2024
Boosting Conversational Question Answering with Fine-Grained
  Retrieval-Augmentation and Self-Check
Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check
Linhao Ye
Zhikai Lei
Jia-Peng Yin
Qin Chen
Jie Zhou
Liang He
3DVRALM
183
31
0
27 Mar 2024
Cross-lingual Contextualized Phrase Retrieval
Cross-lingual Contextualized Phrase Retrieval
Huayang Li
Deng Cai
Zhi Qu
Qu Cui
Hidetaka Kamigaito
Lemao Liu
Taro Watanabe
159
1
0
25 Mar 2024
Language Models Can Reduce Asymmetry in Information Markets
Language Models Can Reduce Asymmetry in Information Markets
Nasim Rahaman
Martin Weiss
Manuel Wüthrich
Yoshua Bengio
Erran L. Li
C. Pal
Bernhard Schölkopf
201
8
0
21 Mar 2024
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language
  Models through Question Complexity
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
Soyeong Jeong
Jinheon Baek
Sukmin Cho
Sung Ju Hwang
Jong C. Park
RALM
369
339
0
21 Mar 2024
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
Yuren Mao
Xuemei Dong
Wenyi Xu
Yunjun Gao
Bin Wei
Ying Zhang
196
18
0
21 Mar 2024
TAG: Guidance-free Open-Vocabulary Semantic Segmentation
TAG: Guidance-free Open-Vocabulary Semantic Segmentation
Yasufumi Kawano
Yoshimitsu Aoki
VLM
161
5
0
17 Mar 2024
DiPaCo: Distributed Path Composition
DiPaCo: Distributed Path Composition
Arthur Douillard
Qixuang Feng
Andrei A. Rusu
A. Kuncoro
Yani Donchev
Rachita Chhaparia
Ionel Gog
MarcÁurelio Ranzato
Jiajun Shen
Arthur Szlam
MoE
235
6
0
15 Mar 2024
RAFT: Adapting Language Model to Domain Specific RAG
RAFT: Adapting Language Model to Domain Specific RAG
Tianjun Zhang
Shishir G. Patil
Naman Jain
Sheng Shen
Matei A. Zaharia
Ion Stoica
Joseph E. Gonzalez
RALM
316
296
0
15 Mar 2024
DRAGIN: Dynamic Retrieval Augmented Generation based on the Information
  Needs of Large Language Models
DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024
Weihang Su
Yichen Tang
Jiaxin Mao
Zhijing Wu
Yiqun Liu
3DVRALMAI4TSSyDa
321
49
0
15 Mar 2024
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal
  Learning with Missing Modalities and Data Scarcity
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity
Zhuo Zhi
Ziquan Liu
M. Elbadawi
Adam Daneshmend
Mine Orlu
Abdul Basit
Andreas Demosthenous
Miguel R. D. Rodrigues
282
4
0
14 Mar 2024
Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D
  Prior
Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D PriorComputer Vision and Pattern Recognition (CVPR), 2024
Cheng Chen
Xiaofeng Yang
Fan Yang
Chengzeng Feng
Zhoujie Fu
Chuan-Sheng Foo
Guosheng Lin
Fayao Liu
295
27
0
14 Mar 2024
Previous
123...8910...161718
Next