Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 248 papers shown

Title
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 28 30 0 17 Jun 2024
Are Large Language Models True Healthcare Jacks-of-All-Trades? Benchmarking Across Health Professions Beyond Physician Exams Zheheng Luo Chenhan Yuan Qianqian Xie Sophia Ananiadou ELM AI4MH LM&MA 41 0 0 17 Jun 2024
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities Zhonghao Li Xuming Hu Aiwei Liu Kening Zheng S. Huang Hui Xiong RALM 115 8 0 17 Jun 2024
CRAG -- Comprehensive RAG Benchmark Xiao Yang Kai Sun Hao Xin Yushi Sun Nikita Bhalla ... Nirav Shah Rakesh Wanga Anuj Kumar Wen-tau Yih Xin Luna Dong 18 22 0 07 Jun 2024
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways Zehang Deng Yongjian Guo Changzhou Han Wanlun Ma Junwu Xiong Sheng Wen Yang Xiang 42 19 0 04 Jun 2024
DAFNet: Dynamic Auxiliary Fusion for Sequential Model Editing in Large Language Models Taolin Zhang Qizhou Chen Dongyang Li Chengyu Wang Xiaofeng He Longtao Huang Hui Xue Junyuan Huang CLL KELM 40 3 0 31 May 2024
Evaluating the External and Parametric Knowledge Fusion of Large Language Models Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu ... Yasheng Wang Lifeng Shang Qun Liu Yong-jin Liu Ruiming Tang KELM 33 4 0 29 May 2024
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution Minghan Li Xilun Chen Ari Holtzman Beidi Chen Jimmy Lin Wen-tau Yih Xi Victoria Lin RALM BDL 108 10 0 29 May 2024
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis Yue Yang Mona Gandhi Yufei Wang Yifan Wu Michael S. Yao Christopher Callison-Burch James C. Gee Mark Yatskar 38 3 0 23 May 2024
Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards Xiaoyu Yang Jie Lu Enshui Yu VLM 35 1 0 22 May 2024
SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation Abhishek Divekar Greg Durrett 26 5 0 16 May 2024
LMD3: Language Model Data Density Dependence John Kirchenbauer Garrett Honke Gowthami Somepalli Jonas Geiping Daphne Ippolito Katherine Lee Tom Goldstein David Andre 22 6 0 10 May 2024
Can large language models understand uncommon meanings of common words? Jinyang Wu Feihu Che Xinxin Zheng Shuai Zhang Ruihan Jin Shuai Nie Pengpeng Shao Jianhua Tao 29 1 0 09 May 2024
Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding Zheng Zhao Emilio Monti Jens Lehmann H. Assem 42 21 0 04 May 2024
Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents Sneha Singhania Simon Razniewski G. Weikum RALM 34 1 0 04 May 2024
FLAME: Factuality-Aware Alignment for Large Language Models Sheng-Chieh Lin Luyu Gao Barlas Oğuz Wenhan Xiong Jimmy Lin Wen-tau Yih Xilun Chen HILM 26 14 0 02 May 2024
Automated Construction of Theme-specific Knowledge Graphs Linyi Ding Sizhe Zhou Jinfeng Xiao Jiawei Han 35 8 0 29 Apr 2024
Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models Zhongzhen Huang Kui Xue Yongqi Fan Linjie Mu Ruoyu Liu Tong Ruan Shaoting Zhang Xiaofan Zhang LM&MA RALM 30 5 0 27 Apr 2024
Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Vittoria Dentella Fritz Guenther Evelina Leivada ELM 22 1 0 23 Apr 2024
U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI Tanja Sarcevic Alicja Karlowicz Rudolf Mayer Ricardo A. Baeza-Yates Andreas Rauber 40 5 0 22 Apr 2024
AmbigDocs: Reasoning across Documents on Different Entities under the Same Name Yoonsang Lee Xi Ye Eunsol Choi 38 5 0 18 Apr 2024
Unifying Bias and Unfairness in Information Retrieval: A Survey of Challenges and Opportunities with Large Language Models Sunhao Dai Chen Xu Shicheng Xu Liang Pang Zhenhua Dong Jun Xu 42 60 0 17 Apr 2024
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory Ali Modarressi Abdullatif Köksal Ayyoob Imani Mohsen Fayyaz Hinrich Schütze KELM 96 9 0 17 Apr 2024
Fewer Truncations Improve Language Modeling Hantian Ding Zijian Wang Giovanni Paolini Varun Kumar Anoop Deoras Dan Roth Stefano Soatto 56 13 0 16 Apr 2024
RAR-b: Reasoning as Retrieval Benchmark Chenghao Xiao G. Thomas Al Moubayed LRM RALM 29 8 0 09 Apr 2024
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance Vishaal Udandarao Ameya Prabhu Adhiraj Ghosh Yash Sharma Philip H. S. Torr Adel Bibi Samuel Albanie Matthias Bethge VLM 118 43 0 04 Apr 2024
AI and the Problem of Knowledge Collapse Andrew J. Peterson 33 16 0 04 Apr 2024
On Large Language Models' Hallucination with Regard to Known Facts Che Jiang Biqing Qi Xiangyu Hong Dayuan Fu Yang Cheng Fandong Meng Mo Yu Bowen Zhou Jie Zhou HILM LRM 31 17 0 29 Mar 2024
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction Yuren Mao Xuemei Dong Wenyi Xu Yunjun Gao Bin Wei Ying Zhang 28 9 0 21 Mar 2024
WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Models Hichem Ammar Khodja Frédéric Béchet Quentin Brabant Alexis Nasr Gwénolé Lecorvé HILM KELM SyDa 16 6 0 21 Mar 2024
Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors Guanghua Li Wensheng Lu Wei Zhang Defu Lian Kezhong Lu Rui Mao Kai Shu Hao Liao HILM 14 4 0 14 Mar 2024
Unfamiliar Finetuning Examples Control How Language Models Hallucinate Katie Kang Eric Wallace Claire Tomlin Aviral Kumar Sergey Levine HILM LRM 22 49 0 08 Mar 2024
Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models Martin Riddell Ansong Ni Arman Cohan ELM 29 28 0 06 Mar 2024
Reliable, Adaptable, and Attributable Language Models with Retrieval Akari Asai Zexuan Zhong Danqi Chen Pang Wei Koh Luke Zettlemoyer Hanna Hajishirzi Wen-tau Yih KELM RALM 36 53 0 05 Mar 2024
KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection Yuexin Li Chengyu Huang Shumin Deng Mei Lin Lock Tri Cao Nay Oo Hoon Wei Lim Bryan Hooi 39 16 0 04 Mar 2024
Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge Heydar Soudani Evangelos Kanoulas Faegheh Hasibi 21 27 0 03 Mar 2024
CR-LT-KGQA: A Knowledge Graph Question Answering Dataset Requiring Commonsense Reasoning and Long-Tail Knowledge Willis Guo Armin Toroghi Scott Sanner RALM 25 5 0 03 Mar 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 110 215 0 29 Feb 2024
Corpus-Steered Query Expansion with Large Language Models Yibin Lei Yu Cao Tianyi Zhou Tao Shen Andrew Yates 22 2 0 28 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
FairBelief -- Assessing Harmful Beliefs in Language Models Mattia Setzu Marta Marchiori Manerba Pasquale Minervini Debora Nozza 18 0 0 27 Feb 2024
Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup Tristan Kenneweg Philip Kenneweg Barbara Hammer 3DV 40 3 0 26 Feb 2024
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering Zihan Zhang Meng Fang Ling-Hao Chen RALM 54 11 0 26 Feb 2024
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study Tianjie Ju Weiwei Sun Wei Du Xinwei Yuan Zhaochun Ren Gongshen Liu KELM 31 23 0 25 Feb 2024
On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe Ningyu Xu Qi Zhang Menghan Zhang Peng Qian Xuanjing Huang LRM 59 3 0 22 Feb 2024
Can Language Models Act as Knowledge Bases at Scale? Qiyuan He Yizhong Wang Wenya Wang KELM LRM 24 7 0 22 Feb 2024
Kuaiji: the First Chinese Accounting Large Language Model Jiayuan Luo Songhua Yang Xiaoling Qiu Panyu Chen Yufei Nai Wenxuan Zeng Wentao Zhang Xinke Jiang RALM ALM 30 1 0 21 Feb 2024
ActiveRAG: Revealing the Treasures of Knowledge via Active Learning Zhipeng Xu Zhenghao Liu Yibin Liu Chenyan Xiong Yukun Yan Shuo Wang Shi Yu Zhiyuan Liu Ge Yu VLM 28 3 0 21 Feb 2024
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models Seiji Maekawa Hayate Iso Sairam Gurajada Nikita Bhutani RALM KELM 23 14 0 21 Feb 2024
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies Xiao Ye Andrew Wang Jacob Choi Yining Lu Shreya Sharma Lingfeng Shen Vijay Tiyyala Nicholas Andrews Daniel Khashabi ELM 31 8 0 19 Feb 2024