26
0

Novel Change of Measure Inequalities and PAC-Bayesian Bounds

Abstract

PAC-Bayesian theory has received a growing attention in the machine learning community. Our work extends the PAC-Bayesian theory by introducing several novel change of measure inequalities for two families of divergences: ff-divergences and α\alpha-divergences. First, we show how the variational representation for ff-divergences leads to novel change of measure inequalities. Second, we propose a multiplicative change of measure inequality for α\alpha-divergences, which leads to tighter bounds under some technical conditions. Finally, we present several PAC-Bayesian bounds for various classes of random variables, by using our novel change of measure inequalities.

View on arXiv
Comments on this paper