The Complexity of Estimating Rényi Entropy

Abstract
It was recently shown that estimating the Shannon entropy of a discrete -symbol distribution requires samples, a number that grows near-linearly in the support size. In many applications can be replaced by the more general R\'enyi entropy of order , . We determine the number of samples needed to estimate . for all , showing that requires a super-linear, roughly samples, noninteger requires a near-linear samples, but, perhaps surprisingly, integer requires only samples. In particular, estimating , which arises in security, DNA reconstruction, closeness testing, and other applications, requires only samples. The estimators achieving these bounds are simple and run in time linear in the number of samples.
View on arXivComments on this paper