v1v2v3 (latest)

Measures of Entropy from Data Using Infinitely Divisible Kernels

11 November 2012

Luis G. Sanchez Giraldo

M. Rao

José C. Príncipe

ArXiv (abs)PDF HTML

Papers citing "Measures of Entropy from Data Using Infinitely Divisible Kernels"

50 / 62 papers shown

Title
From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning Chen Shani Dan Jurafsky Yann LeCun Ravid Shwartz-Ziv 206 0 0 21 May 2025
Memorization-Compression Cycles Improve Generalization Fangyuan Yu 71 0 0 13 May 2025
DocVXQA: Context-Aware Visual Explanations for Document Question Answering Mohamed Ali Souibgui Changkyu Choi Andrey Barsky Kangsoo Jung Ernest Valveny Dimosthenis Karatzas 142 0 0 12 May 2025
Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering Xinyu Tang Xiaolei Wang Zhihao Lv Yingqian Min Wayne Xin Zhao Binbin Hu Ziqi Liu Qing Cui LRM 153 9 0 14 Mar 2025
Understanding the role of autoencoders for stiff dynamical systems using information theory Vijayamanikandan Vijayarangan Harshavardhana A. Uranakara Francisco E. Hernández-Pérez Hong G. Im SyDa AI4CE 84 1 0 08 Mar 2025
Does Representation Matter? Exploring Intermediate Layers in Large Language Models Oscar Skean Md Rifat Arefin Yann LeCun Ravid Shwartz-Ziv 134 12 0 12 Dec 2024
Conditional Vendi Score: An Information-Theoretic Approach to Diversity Evaluation of Prompt-based Generative Models Mohammad Jalali Azim Ospanov Amin Gohari Farzan Farnia EGVM 96 4 0 05 Nov 2024
Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning Md Rifat Arefin G. Subbaraj Nicolas Angelard-Gontier Yann LeCun Irina Rish Ravid Shwartz-Ziv C. Pal LRM 494 2 0 04 Nov 2024
An Analysis of Minimum Error Entropy Loss Functions in Wireless Communications Rumeshika Pallewela Eslam Eldeeb Hirley Alves 67 0 0 25 Sep 2024
Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint Sixiang Chen Tian-Chun Ye Peng Sun Zhaohu Xing Yunlong Lin Lei Zhu DiffM 76 12 0 24 Sep 2024
Learning to Embed Distributions via Maximum Kernel Entropy Oleksii Kachaiev Stefano Recanatesi OOD 120 0 0 01 Aug 2024
Discovering Common Information in Multi-view Data Qi Zhang Mingfei Lu Shujian Yu Jingmin Xin Badong Chen 79 7 0 21 Jun 2024
Large Language Model Pruning Hanjuan Huang Hao-Jia Song H. Pao 117 0 0 24 May 2024
Differentiable Information Bottleneck for Deterministic Multi-view Clustering Xiaoqiang Yan Zhixiang Jin Fengshou Han Yangdong Ye 65 4 0 23 Mar 2024
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness Xiaoyun Xu Shujian Yu Jingzheng Wu S. Picek AAML 115 0 0 08 Dec 2023
FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learning Oscar Skean Aayush Dhakal Nathan Jacobs Luis Gonzalo Sánchez Giraldo 70 0 0 04 Oct 2023
Information Flow in Self-Supervised Learning Zhiyuan Tan Jingqin Yang Weiran Huang Yang Yuan Yifan Zhang SSL 83 14 0 29 Sep 2023
MOLE: MOdular Learning FramEwork via Mutual Information Maximization Tianchao Li Yulong Pei 72 0 0 15 Aug 2023
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study Luis P. Silvestrin Shujian Yu Mark Hoogendoorn OOD 97 1 0 17 Jul 2023
Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization Shuo Ye Shujian Yu W. Hou Yu Wang Xinge You OOD 77 11 0 08 Jun 2023
The Representation Jensen-Shannon Divergence J. Hoyos-Osorio Santiago Posso-Murillo L. S. Giraldo 159 8 0 25 May 2023
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective Yuxin Dong Tieliang Gong Hao Chen Chen Li 62 3 0 02 May 2023
Local Intrinsic Dimensional Entropy Rohan Ghosh Mehul Motani 58 2 0 05 Apr 2023
Higher-order Organization in the Human Brain from Matrix-Based Rényi's Entropy Qiang Li Shujian Yu Kristoffer Hougaard Madsen Vince D. Calhoun A. Iraji 66 10 0 21 Mar 2023
MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation Roy Miles M. K. Yucel Bruno Manganelli Albert Saà-Garriga VOS 81 25 0 14 Mar 2023
DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies Oscar Skean J. Hoyos-Osorio A. Brockmeier L. S. Giraldo 80 13 0 19 Jan 2023
Causal Recurrent Variational Autoencoder for Medical Time Series Generation Hongming Li Shujian Yu José C. Príncipe CML BDL MedIm 79 52 0 16 Jan 2023
CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis Kaizhong Zheng Shujian Yu Badong Chen CML 125 33 0 04 Jan 2023
Robust and Fast Measure of Information via Low-rank Representation Yuxin Dong Tieliang Gong Shujian Yu Hong Chen Chen Li 64 1 0 30 Nov 2022
Sum-of-Squares Relaxations for Information Theory and Variational Inference Francis R. Bach 67 12 0 27 Jun 2022
Optimal Randomized Approximations for Matrix based Renyi's Entropy Yuxin Dong Tieliang Gong Shujian Yu Chen Li 71 8 0 16 May 2022
BrainIB: Interpretable Brain Network-based Psychiatric Diagnosis with Graph Information Bottleneck Kaizhong Zheng Shujian Yu Baojuan Li Robert Jenssen Badong Chen AI4MH 89 10 0 07 May 2022
Multi-view Information Bottleneck Without Variational Approximation Qi Zhang Shujian Yu J. Xin Badong Chen 67 10 0 22 Apr 2022
R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction Yu Wang Shuo Ye Shujian Yu Xinge You 79 16 0 21 Apr 2022
HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels C. Sarvani Mrinmoy Ghorai S. Dubey S. H. Shabbeer Basha VLM 99 39 0 22 Feb 2022
Theoretical Analysis of Deep Neural Networks in Physical Layer Communication Jing Liu Haitao Zhao Dongtang Ma Kai Mei Jibo Wei 67 4 0 21 Feb 2022
Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing Hongming Li Shujian Yu José C. Príncipe 56 6 0 07 Feb 2022
Computationally Efficient Approximations for Matrix-based Renyi's Entropy Tieliang Gong Yuxin Dong Shujian Yu B. Dong 118 3 0 27 Dec 2021
The Representation Jensen-Rényi Divergence J. Hoyos-Osorio Oscar Skean A. Brockmeier L. S. Giraldo 47 2 0 02 Dec 2021
Information Theoretic Representation Distillation Roy Miles Adrian Lopez-Rodriguez K. Mikolajczyk MQ 103 22 0 01 Dec 2021
Gated Information Bottleneck for Generalization in Sequential Environments Francesco Alesiani Shujian Yu Xi Yu OOD AAML 58 13 0 12 Oct 2021
Information Theoretic Structured Generative Modeling Bo Hu Shujian Yu José C. Príncipe SyDa DRL 51 0 0 12 Oct 2021
Estimating Rényi's $α$ -Cross-Entropies in a Matrix-Based Way I. Sledge José C. Príncipe 102 0 0 24 Sep 2021
Kernel PCA with the Nyström method Fredrik Hallgren 46 3 0 12 Sep 2021
Opening the Black Box of Deep Neural Networks in Physical Layer Communication Jun Liu Haitao Zhao Dongtang Ma Kai Mei Jibo Wei AI4CE 63 3 0 02 Jun 2021
Understanding Neural Networks with Logarithm Determinant Entropy Estimator Zhanghao Zhouyin Ding Liu FAtt 49 9 0 08 May 2021
InfoNEAT: Information Theory-based NeuroEvolution of Augmenting Topologies for Side-channel Analysis R. Acharya F. Ganji Domenic Forte AAML 116 25 0 30 Apr 2021
Information flows of diverse autoencoders Sungyeop Lee Junghyo Jo 51 11 0 15 Feb 2021
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional Xi Yu Shujian Yu José C. Príncipe AAML 74 27 0 31 Jan 2021
Measuring Dependence with Matrix-based Entropy Functional Shujian Yu Francesco Alesiani Xi Yu Robert Jenssen José C. Príncipe 57 24 0 25 Jan 2021