Entropy Law: The Story Behind Data Compression and LLM Performance

Entropy Law: The Story Behind Data Compression and LLM Performance

9 July 2024

Hao Wang

Wei Guo

Papers citing "Entropy Law: The Story Behind Data Compression and LLM Performance"

17 / 17 papers shown

Title
The Rise of Small Language Models in Healthcare: A Comprehensive Survey Muskan Garg Shaina Raza Shebuti Rayana Xingyi Liu Sunghwan Sohn LM&MA AILaw 87 0 0 23 Apr 2025
MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space Yicheng Chen Yining Li Kai Hu Zerun Ma Haochen Ye Kai Chen 27 0 0 18 Apr 2025
Advancing MAPF towards the Real World: A Scalable Multi-Agent Realistic Testbed (SMART) Jingtian Yan Zhifei Li William Kang Yulun Zhang Stephen Smith Jiaoyang Li 38 0 0 03 Mar 2025
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation Yanzhou Pan Huawei Lin Yide Ran Jiamin Chen Xiaodong Yu Weijie Zhao Denghui Zhang Zhaozhuo Xu 40 0 0 02 Mar 2025
A Universal Framework for Compressing Embeddings in CTR Prediction Kefan Wang Hao Wang Kenan Song Wei Guo Kai Cheng Z. Li Y. Liu Defu Lian Enhong Chen 59 1 0 24 Feb 2025
Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation Hao Wang Wei Guo L. Zhang Jin Yao Chin Yufei Ye Huifeng Guo Y. Liu Defu Lian Ruiming Tang Enhong Chen 56 1 0 20 Feb 2025
The Representation and Recall of Interwoven Structured Knowledge in LLMs: A Geometric and Layered Analysis Ge Lei Samuel J. Cooper KELM 47 0 0 15 Feb 2025
$FuXi-$\alpha$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer$ FuXi- $\alpha$ : Scaling Recommendation Model with Feature Interaction Enhanced Transformer Yufei Ye Wei Guo Jin Yao Chin Hao Wang Hong Zhu ... Yuyang Ye Y. Liu Ruiming Tang Defu Lian Enhong Chen 89 2 0 05 Feb 2025
Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction Xiang Xu Hao Wang Wei Guo L. Zhang Wanshan Yang R. Yu Y. Liu Defu Lian Enhong Chen 33 5 0 22 Nov 2024
Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model Wenjia Xie Hao Wang L. Zhang Rui Zhou Defu Lian Enhong Chen DiffM 33 3 0 31 Oct 2024
DisenGCD: A Meta Multigraph-assisted Disentangled Graph Learning Framework for Cognitive Diagnosis Shangshang Yang Mingyang Chen Ziwen Wang Xiaoshan Yu Panpan Zhang Haiping Ma Xingyi Zhang 22 0 0 23 Oct 2024
Rethinking Data Selection at Scale: Random Selection is Almost All You Need Tingyu Xia Bowen Yu K. Dang An Yang Yuan Wu Yuan Tian Yi-Ju Chang Junyang Lin ALM 49 3 0 12 Oct 2024
MDAP: A Multi-view Disentangled and Adaptive Preference Learning Framework for Cross-Domain Recommendation Junxiong Tong Mingjia Yin Hao Wang Qiushi Pan Defu Lian Enhong Chen 27 2 0 08 Oct 2024
Bridging User Dynamics: Transforming Sequential Recommendations with Schrödinger Bridge and Diffusion Models Wenjia Xie Rui Zhou Hao Wang Tingjia Shen Enhong Chen DiffM 31 9 0 30 Aug 2024
Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation Hao Wang Yongqiang Han Kefan Wang Kai Cheng Zhen Wang Wei Guo Yong Liu Defu Lian Enhong Chen 33 2 0 21 Aug 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 159 437 0 02 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022