HTLM: Hyper-Text Pre-Training and Prompting of Language Models

14 July 2021

Luke Zettlemoyer

Papers citing "HTLM: Hyper-Text Pre-Training and Prompting of Language Models"

50 / 56 papers shown

Title
When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars Rei Higuchi Ryotaro Kawata Naoki Nishikawa Kazusato Oko Shoichiro Yamaguchi Sosuke Kobayashi Seiya Tokui K. Hayashi Daisuke Okanohara Taiji Suzuki AI4CE 145 1 0 24 Apr 2025
MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model Chengze Zhang Changshan Li Shiyang Gao 158 0 0 03 Jan 2025
The Effects of Hallucinations in Synthetic Training Data for Relation Extraction Steven Rogulsky Nicholas Popovic Michael Färber HILM 91 3 0 10 Oct 2024
HySem: A context length optimized LLM pipeline for unstructured tabular extraction Narayanan PP A. P. N. Iyer 122 1 0 18 Aug 2024
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 123 3 0 11 Jul 2024
Tokenization Falling Short: The Curse of Tokenization Yekun Chai Yewei Fang Qiwei Peng Xuhong Li 111 8 0 17 Jun 2024
Leveraging Large Language Models for Web Scraping Aman Ahluwalia Suhrud Wani 68 10 0 12 Jun 2024
StablePT: Towards Stable Prompting for Few-shot Learning via Input Separation Xiaoming Liu Chen Liu Zhaohan Zhang Chengzhengxu Li Longtian Wang Y. Lan Chao Shen VLM 111 5 0 30 Apr 2024
Tur[k]ingBench: A Challenge Benchmark for Web Agents Kevin Xu Yeganeh Kordi Kate Sanders Yizhong Wang Adam Byerly Kate Sanders Adam Byerly Jingyu Zhang Benjamin Van Durme Daniel Khashabi LLMAG 272 13 0 18 Mar 2024
Combining Language and Graph Models for Semi-structured Information Extraction on the Web Zhi Hong Kyle Chard Ian Foster 55 3 0 21 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 158 102 0 08 Feb 2024
ScreenAI: A Vision-Language Model for UI and Infographics Understanding Gilles Baechler Srinivas Sunkara Maria Wang Fedir Zubach Hassan Mansoor Vincent Etter Victor Carbune Jason Lin Jindong Chen Abhanshu Sharma 284 77 0 07 Feb 2024
Document Structure in Long Document Transformers Jan Buchmann Max Eichler Jan-Micha Bodensohn Ilia Kuznetsov Iryna Gurevych 68 3 0 31 Jan 2024
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification Vidit Jain Mukund Rungta Yuchen Zhuang Yue Yu Zeyu Wang Mu Gao Jeffrey Skolnick Chao Zhang 107 2 0 24 Jan 2024
TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning Yuan Sui Jiaru Zou Mengyu Zhou Xinyi He Lun Du Shi Han Dongmei Zhang LRM LMTD 94 35 0 14 Dec 2023
Self-Infilling Code Generation Lin Zheng Jianbo Yuan Zhi Zhang Hongxia Yang Lingpeng Kong 135 3 0 29 Nov 2023
PixT3: Pixel-based Table-To-Text Generation Iñigo Alonso Eneko Agirre Mirella Lapata LMTD 151 8 0 16 Nov 2023
Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis Kexin Chen Junyou Li Kunyi Wang Yuyang Du Jiahui Yu ... Jianzhang Pan Yi Huang Qun Fang Pheng Ann Heng Guangyong Chen 139 15 0 16 Nov 2023
Towards Concept-Aware Large Language Models Chen Shani Jilles Vreeken Dafna Shahaf LRM 74 8 0 03 Nov 2023
Automatic Logical Forms improve fidelity in Table-to-Text generation Iñigo Alonso Eneko Agirre LMTD 121 4 0 26 Oct 2023
Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education M. Abedi Ibrahem Alshybani M. Shahadat M. Murillo 143 17 0 09 Sep 2023
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning Alexander Hanbo Li Mingyue Shang Evangelia Spiliopoulou Jie Ma Patrick Ng ... William Yang Wang Kathleen McKeown Vittorio Castelli Dan Roth Bing Xiang 88 3 0 10 Aug 2023
Generative Models as a Complex Systems Science: How can we make sense of large language model behavior? Ari Holtzman Peter West Luke Zettlemoyer AI4CE 153 16 0 31 Jul 2023
Schema-Driven Information Extraction from Heterogeneous Tables Fan Bai Junmo Kang Gabriel Stanovsky Dayne Freitag Alan Ritter LMTD 128 19 0 23 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 259 127 0 19 May 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 152 617 0 07 Mar 2023
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset Hugo Laurenccon Lucile Saulnier Thomas Wang Christopher Akiki Albert Villanova del Moral ... Violette Lepercq Suzana Ilić Margaret Mitchell Sasha Luccioni Yacine Jernite AI4CE AILaw 106 185 0 07 Mar 2023
KILM: Knowledge Injection into Encoder-Decoder Language Models Yan Xu Mahdi Namazifar Devamanyu Hazarika Aishwarya Padmakumar Yang Liu Dilek Z. Hakkani-Tür KELM 94 28 0 17 Feb 2023
Transformer models: an introduction and catalog X. Amatriain Ananth Sankar Jie Bing Praveen Kumar Bodigutla Timothy J. Hazen Michaeel Kazi 254 59 0 12 Feb 2023
Toolformer: Language Models Can Teach Themselves to Use Tools Timo Schick Jane Dwivedi-Yu Roberto Dessì Roberta Raileanu Maria Lomeli Luke Zettlemoyer Nicola Cancedda Thomas Scialom SyDa RALM 301 2,150 0 09 Feb 2023
BARTSmiles: Generative Masked Language Models for Molecular Representations Gayane Chilingaryan Hovhannes Tamoyan Ani Tevosyan N. Babayan L. Khondkaryan Karen Hambardzumyan Zaven Navoyan Hrant Khachatrian Armen Aghajanyan SSL 149 31 0 29 Nov 2022
QueryForm: A Simple Zero-shot Form Entity Query Framework Zifeng Wang Zizhao Zhang Jacob Devlin Chen-Yu Lee Guolong Su Hao Zhang Jennifer Dy Vincent Perot Tomas Pfister 84 8 0 14 Nov 2022
An Inclusive Notion of Text Ilia Kuznetsov Iryna Gurevych 93 0 0 10 Nov 2022
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 278 94 0 14 Oct 2022
Understanding HTML with Large Language Models Izzeddin Gur Ofir Nachum Yingjie Miao Mustafa Safdari Austin Huang Aakanksha Chowdhery Sharan Narang Noah Fiedel Aleksandra Faust AI4CE 287 78 0 08 Oct 2022
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding Kenton Lee Mandar Joshi Iulia Turc Hexiang Hu Fangyu Liu Julian Martin Eisenschlos Urvashi Khandelwal Peter Shaw Ming-Wei Chang Kristina Toutanova CLIP VLM 384 326 0 07 Oct 2022
WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphs Hoang Thang Ta Abu Bakar Siddiqur Rahman Navonil Majumder Amir Hussain Lotfollah Najjar N. Howard Soujanya Poria Alexander Gelbukh 110 14 0 27 Sep 2022
Few-shot Adaptation Works with UnpredicTable Data Jun Shern Chan Michael Pieler Jonathan Jao Jérémy Scheurer Ethan Perez 157 6 0 01 Aug 2022
Efficient Training of Language Models to Fill in the Middle Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman C. McLeavey Jerry Tworek Mark Chen 122 232 0 28 Jul 2022
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents Shunyu Yao Howard Chen John Yang Karthik Narasimhan LLMAG LM&Ro 469 631 0 04 Jul 2022
On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method Zorik Gekhman Nadav Oved Orgad Keller Idan Szpektor Roi Reichart 112 8 0 29 Jun 2022
UL2: Unifying Language Learning Paradigms Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason W. Wei ... Tal Schuster H. Zheng Denny Zhou N. Houlsby Donald Metzler AI4CE 271 327 0 10 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 158 70 0 03 May 2022
A Review on Language Models as Knowledge Bases Badr AlKhamissi Millicent Li Asli Celikyilmaz Mona T. Diab Marjan Ghazvininejad KELM 171 199 0 12 Apr 2022
InCoder: A Generative Model for Code Infilling and Synthesis Daniel Fried Armen Aghajanyan Jessy Lin Sida I. Wang Eric Wallace Freda Shi Ruiqi Zhong Anuj Kumar Luke Zettlemoyer M. Lewis SyDa 197 718 0 12 Apr 2022
LinkBERT: Pretraining Language Models with Document Links Michihiro Yasunaga J. Leskovec Percy Liang KELM 154 390 0 29 Mar 2022
UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL Longxu Dou Yan Gao Mingyang Pan Dingzirui Wang Wanxiang Che Dechen Zhan Jian-Guang Lou 131 30 0 15 Mar 2022
WebFormer: The Web-page Transformer for Structure Information Extraction Qifan Wang Yi Fang Anirudh Ravula Fuli Feng Xiaojun Quan Dongfang Liu ViT 253 75 0 01 Feb 2022
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection Suchin Gururangan Dallas Card Sarah K. Drier E. K. Gade Leroy Z. Wang Zeyu Wang Luke Zettlemoyer Noah A. Smith 348 89 0 25 Jan 2022
CM3: A Causal Masked Multimodal Model of the Internet Armen Aghajanyan Po-Yao (Bernie) Huang Candace Ross Vladimir Karpukhin Hu Xu ... Dmytro Okhonko Mandar Joshi Gargi Ghosh M. Lewis Luke Zettlemoyer 183 165 0 19 Jan 2022