Large Language Models Struggle to Learn Long-Tail Knowledge

15 November 2022

Papers citing "Large Language Models Struggle to Learn Long-Tail Knowledge"

50 / 248 papers shown

Title
LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering Qingfei Zhao Ruobing Wang Yukuo Cen Daren Zha Shicheng Tan Yuxiao Dong Jie Tang RALM 31 8 0 23 Oct 2024
Scalable Influence and Fact Tracing for Large Language Model Pretraining Tyler A. Chang Dheeraj Rajagopal Tolga Bolukbasi Lucas Dixon Ian Tenney TDI 28 0 0 22 Oct 2024
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion Denitsa Saynova Lovisa Hagström Moa Johansson Richard Johansson Marco Kuhlmann HILM 34 0 0 18 Oct 2024
RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards Xinze Li Sen Mei Zhenghao Liu Yukun Yan Shuo Wang ... H. Chen Ge Yu Zhiyuan Liu Maosong Sun Chenyan Xiong 39 6 0 17 Oct 2024
Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models Jiatao Li Xinyu Hu Xunjian Yin Xiaojun Wan RALM 48 0 0 17 Oct 2024
Telco-DPR: A Hybrid Dataset for Evaluating Retrieval Models of 3GPP Technical Specifications Thaina Saraiva Marco Sousa Pedro Vieira António Rodrigues 19 0 0 15 Oct 2024
A Multi-LLM Orchestration Engine for Personalized, Context-Rich Assistance Sumedh Rasal 15 0 0 13 Oct 2024
NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models Zheng Yi Ho Siyuan Liang Sen Zhang Yibing Zhan Dacheng Tao 26 1 0 11 Oct 2024
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning Tingchen Fu Mrinank Sharma Philip H. S. Torr Shay B. Cohen David M. Krueger Fazl Barez AAML 42 7 0 11 Oct 2024
Large Language Models in Qualitative Research: Can We Do the Data Justice? Hope Schroeder Marianne Aubin Le Quéré Casey Randazzo David Mimno Sarita Schoenebeck 13 0 0 09 Oct 2024
Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models M. Farahani Richard Johansson RALM 21 2 0 07 Oct 2024
ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection Yibo Yan Shen Wang Jiahao Huo Hang Li B. Li ... Kun Wang Hui Xiong Philip S. Yu Xuming Hu Qingsong Wen LRM 25 13 0 06 Oct 2024
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024
Reward-RAG: Enhancing RAG with Reward Driven Supervision Thang Nguyen Peter Chin Yu-Wing Tai RALM 32 4 0 03 Oct 2024
Mitigating Memorization In Language Models Mansi Sakarvadia Aswathy Ajith Arham Khan Nathaniel Hudson Caleb Geniesse Kyle Chard Yaoqing Yang Ian Foster Michael W. Mahoney KELM MU 44 0 0 03 Oct 2024
Quantifying Generalization Complexity for Large Language Models Zhenting Qi Hongyin Luo Xuliang Huang Zhuokai Zhao Yibo Jiang Xiangjun Fan Himabindu Lakkaraju James Glass LRM ELM 26 5 0 02 Oct 2024
Reasoning Elicitation in Language Models via Counterfactual Feedback Alihan Hüyük Xinnuo Xu Jacqueline Maasch Aditya V. Nori Javier González ReLM LRM 62 1 0 02 Oct 2024
Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models Joseph Lee Shu Yang Jae Young Baik Xiaoxi Liu Zhen Tan ... Zixuan Wen Bojian Hou D. Duong-Tran Tianlong Chen Li Shen 44 1 0 02 Oct 2024
Decoding Large-Language Models: A Systematic Overview of Socio-Technical Impacts, Constraints, and Emerging Questions Zeyneb N. Kaya Souvick Ghosh 35 0 0 25 Sep 2024
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework Lu Chen Ruqing Zhang Jiafeng Guo Yixing Fan Xueqi Cheng 24 2 0 24 Sep 2024
RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph Linxi Wei Guorui Xiao Magdalena Balazinska 31 1 0 22 Sep 2024
Synthetic continued pretraining Zitong Yang Neil Band Shuangping Li Emmanuel Candès Tatsunori Hashimoto CLL SyDa 36 11 0 11 Sep 2024
Retrieval Augmented Correction of Named Entity Speech Recognition Errors Ernest Pusateri Anmol Walia Anirudh Kashi Bortik Bandyopadhyay Nadia Hyder Sayantan Mahinder R. Anantha Daben Liu Sashank Gondala RALM 3DV 26 2 0 09 Sep 2024
Pairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&A K Roth Rushil Gupta Simon Halle Bang Liu RALM 27 0 0 02 Sep 2024
Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models Jinyang Wu Feihu Che Chuyuan Zhang Jianhua Tao Shuai Zhang Pengpeng Shao 28 2 0 24 Aug 2024
Ancient Wisdom, Modern Tools: Exploring Retrieval-Augmented LLMs for Ancient Indian Philosophy Priyanka Mandikal RALM VLM 40 0 0 21 Aug 2024
Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models Amey Hengle Prasoon Bajpai Soham Dan Tanmoy Chakraborty LRM 21 2 0 19 Aug 2024
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability Jiri Hron Laura J. Culp Gamaleldin F. Elsayed Rosanne Liu Ben Adlam ... T. Warkentin Lechao Xiao Kelvin Xu Jasper Snoek Simon Kornblith 21 1 0 14 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Page-Caccia Haokun Liu Tianlong Chen Mohit Bansal Leshem Choshen Alessandro Sordoni MoMe 38 21 0 13 Aug 2024
KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models Ruizhe Zhang Yongxin Xu Yuzhen Xiao Runchuan Zhu Xinke Jiang Xu Chu Junfeng Zhao Yasha Wang 32 2 0 06 Aug 2024
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons Yifei Wang Yuheng Chen Wanting Wen Yu Sheng Linjing Li D. Zeng KELM 28 5 0 06 Aug 2024
Knowledge Prompting: How Knowledge Engineers Use Large Language Models Elisavet Koutsiana Johanna Walker Michelle Nwachukwu Albert Meroño-Peñuela Elena Simperl 32 1 0 02 Aug 2024
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts Youna Kim Hyuhng Joon Kim Cheonbok Park Choonghyun Park Hyunsoo Cho Junyeob Kim Kang Min Yoo Sang-goo Lee Taeuk Kim 23 4 0 02 Aug 2024
MCGMark: An Encodable and Robust Online Watermark for Tracing LLM-Generated Malicious Code Peng Ding Jingyu Wu Qingyuan Zhong Dan Ma Xunliang Cai ... Shi Chen Weizhe Zhang Zibin Zheng Weizhe Zhang Zibin Zheng 27 0 0 02 Aug 2024
GOProteinGNN: Leveraging Protein Knowledge Graphs for Protein Representation Learning Dan Kalifa Uriel Singer Kira Radinsky 32 1 0 31 Jul 2024
Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy? Hao Shen Zihan Li Minqiang Yang Minghui Ni Yongfeng Tao Zhengyang Yu Weihao Zheng Chen Xu Bin Hu AI4MH 18 3 0 25 Jul 2024
Benchmarks as Microscopes: A Call for Model Metrology Michael Stephen Saxon Ari Holtzman Peter West William Yang Wang Naomi Saphra 23 10 0 22 Jul 2024
MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation Marco Simoni Andrea Saracino Vinod Puthuvath Maurco Conti 50 1 0 22 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 47 27 0 22 Jul 2024
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models Yuji Zhang Sha Li Jiateng Liu Pengfei Yu Yi Ren Fung Jing Li Manling Li Heng Ji 29 10 0 10 Jul 2024
Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions Shumaila Javaid R. A. Khalil Nasir Saeed Bin He Mohamed-Slim Alouini 32 8 0 05 Jul 2024
Why does in-context learning fail sometimes? Evaluating in-context learning on open and closed questions Xiang Li Haoran Tang Siyu Chen Ziwei Wang Ryan Chen Marcin Abram LRM 29 1 0 02 Jul 2024
Understanding Transformers via N-gram Statistics Timothy Nguyen 25 9 0 30 Jun 2024
HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability Yanfang Chen Ding Chen Shichao Song Simin Niu Hanyu Wang Zeyun Tang Feiyu Xiong Zhiyu Li 19 0 0 30 Jun 2024
Learning to Correct for QA Reasoning with Black-box LLMs Jaehyung Kim Dongyoung Kim Yiming Yang LRM 38 3 0 26 Jun 2024
Few-shot Personalization of LLMs with Mis-aligned Responses Jaehyung Kim Yiming Yang 37 7 0 26 Jun 2024
A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems Florin Cuconasu Giovanni Trappolini Nicola Tonellotto Fabrizio Silvestri 51 2 0 21 Jun 2024
CodeRAG-Bench: Can Retrieval Augment Code Generation? Zora Zhiruo Wang Akari Asai Xinyan Velocity Yu Frank F. Xu Yiqing Xie Graham Neubig Daniel Fried RALM 67 30 0 20 Jun 2024
R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation Fuda Ye Shuangyin Li Yongqi Zhang L. Chen 30 0 0 19 Jun 2024
Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation Yuhang Zhou Jing Zhu Paiheng Xu Xiaoyu Liu Xiyao Wang Danai Koutra Wei Ai Furong Huang 70 4 0 19 Jun 2024