Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

We study contextual linear bandit problems under uncertainty on features; they are noisy with missing entries. To address the challenges from the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on noise characteristics, which is highly non-intuitive and does not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. We thus propose an algorithm aiming at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound with respect to feature dimension $d$ and time horizon $T$. We demonstrate the proposed algorithm using synthetic and real-world datasets.