ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 0811.3174
98
5
v1v2v3v4 (latest)

Deformed Statistics Formulation of the Information Bottleneck Method

19 November 2008
R. C. Venkatesan
A. Plastino
ArXiv (abs)PDFHTML
Abstract

A candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis. Given a nonadditivity parameter q q q, the role of the \textit{additive duality} of nonadditive statistics (q∗=2−q q^*=2-q q∗=2−q) in relating Tsallis entropies for ranges of the nonadditivity parameter q<1 q < 1 q<1 and q>1 q > 1 q>1 is described. Defining X X X, X~ \tilde X X~, and Y Y Y to be the source alphabet, the compressed reproduction alphabet, and, the \textit{relevance variable} respectively, it is demonstrated that minimization of a generalized IB Lagrangian defined in terms of the nonadditivity parameter q∗ q^* q∗ self-consistently yields the \textit{nonadditive effective distortion measure} to be the \textit{q q q-deformed} generalized Kullback-Leibler divergence: DK−Lq[p(Y∣X)∣∣p(Y∣X~)] D_{K-L}^{q}[p(Y|X)||p(Y|\tilde X)] DK−Lq​[p(Y∣X)∣∣p(Y∣X~)]. This result is achieved without enforcing any \textit{a-priori} assumptions. Finally, it is proven that the nonadditive free energy of the system in q∗ q^* q∗ space is non-negative and convex. These results generalize critical features of the IB method to the case of Tsallis statistics.

View on arXiv
Comments on this paper