Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2310.10638
Cited By
In-context Pretraining: Language Modeling Beyond Document Boundaries
16 October 2023
Weijia Shi
Sewon Min
Maria Lomeli
Chunting Zhou
Margaret Li
Gergely Szilvasy
Rich James
Victoria Lin
Noah A. Smith
Luke Zettlemoyer
Scott Yih
Mike Lewis
LRM
RALM
SyDa
Re-assign community
ArXiv
PDF
HTML
Papers citing
"In-context Pretraining: Language Modeling Beyond Document Boundaries"
14 / 14 papers shown
Title
CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation
Jiahao Li
Weijian Ma
Xueyang Li
Yunzhong Lou
G. Zhou
Xiangdong Zhou
32
0
0
07 May 2025
Yi-Lightning Technical Report
01. AI
:
Alan Wake
Albert Wang
Bei Chen
...
Yuxuan Sha
Zhaodong Yan
Zhiyuan Liu
Zirui Zhang
Zonghong Dai
OSLM
97
3
0
02 Dec 2024
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following
Gabrielle Kaili-May Liu
Bowen Shi
Avi Caciularu
Idan Szpektor
Arman Cohan
58
3
0
30 Oct 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
S. Yu
C. Tang
Bokai Xu
Junbo Cui
Junhao Ran
...
Zhenghao Liu
Shuo Wang
Xu Han
Zhiyuan Liu
Maosong Sun
VLM
37
22
0
14 Oct 2024
DEPTH: Discourse Education through Pre-Training Hierarchically
Zachary Bamberger
Ofek Glick
Chaim Baskin
Yonatan Belinkov
54
0
0
13 May 2024
Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao
Yuanbin Qu
Konrad Staniszewski
Szymon Tworkowski
Wei Liu
Piotr Milo's
Yuxiang Wu
Pasquale Minervini
29
13
0
21 Feb 2024
Resolving Knowledge Conflicts in Large Language Models
Yike Wang
Shangbin Feng
Heng Wang
Weijia Shi
Vidhisha Balachandran
Tianxing He
Yulia Tsvetkov
48
12
0
02 Oct 2023
Pre-Training to Learn in Context
Yuxian Gu
Li Dong
Furu Wei
Minlie Huang
CLIP
LRM
ReLM
106
37
0
16 May 2023
Training Language Models with Memory Augmentation
Zexuan Zhong
Tao Lei
Danqi Chen
RALM
232
126
0
25 May 2022
Training language models to follow instructions with human feedback
Long Ouyang
Jeff Wu
Xu Jiang
Diogo Almeida
Carroll L. Wainwright
...
Amanda Askell
Peter Welinder
Paul Christiano
Jan Leike
Ryan J. Lowe
OSLM
ALM
303
11,730
0
04 Mar 2022
Meta-learning via Language Model In-context Tuning
Yanda Chen
Ruiqi Zhong
Sheng Zha
George Karypis
He He
218
155
0
15 Oct 2021
Dict-BERT: Enhancing Language Model Pre-training with Dictionary
W. Yu
Chenguang Zhu
Yuwei Fang
Donghan Yu
Shuohang Wang
Yichong Xu
Michael Zeng
Meng-Long Jiang
45
64
0
13 Oct 2021
Entity-Based Knowledge Conflicts in Question Answering
Shayne Longpre
Kartik Perisetla
Anthony Chen
Nikhil Ramesh
Chris DuBois
Sameer Singh
HILM
241
236
0
10 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Ofir Press
Noah A. Smith
M. Lewis
242
690
0
27 Aug 2021
1