Asymptotic behaviour of weighted differential entropies in a Bayesian
problem
Consider a Bayesian problem of estimating of probability of success in a series of trials with binary outcomes. We study the asymptotic behaviour of weighted differential entropies for posterior probability density function (PDF) conditional on successes after trials, when . In the first part of work Shannon's differential entropy is considered in three particular cases: is a proportion of ; , where ; either or is a constant. In the first and second cases limiting distribution is Gaussian and the asymptotic of differential entropy is asymptotically Gaussian with corresponding variances. In the third case the limiting distribution in not Gaussian, but still the asymptotic of differential entropy can be found explicitly. Then suppose that one is interested to know whether the coin is fair or not and for large is interested in the true frequency. In other words, one wants to emphasize the parameter value . To do so the concept of weighted differential entropy is used when the frequency is necessary to emphasize. It was found that the weight in suggested form does not change the asymptotic form of Shannon, Renyi, Tsallis and Fisher entropies, but change the constants. The main term in weighted Fisher Information is changed by some constant which depend on distance between the true frequency and the value we want to emphasize.
View on arXiv