4
12A Hoeffding Inequality for Finite State Markov Chains and its Applications to Markovian Bandits
Abstract
This paper develops a Hoeffding inequality for the partial sums , where is an irreducible Markov chain on a finite state space , and is a real-valued function. Our bound is simple, general, since it only assumes irreducibility and finiteness of the state space, and powerful. In order to demonstrate its usefulness we provide two applications in multi-armed bandit problems. The first is about identifying an approximately best Markovian arm, while the second is concerned with regret minimization in the context of Markovian bandits.
View on arXivComments on this paper