Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains. We then show that these results can be applied to establish a non-asymptotic central limit theorem for Temporal Difference (TD) learning with averaging.

@article{srikant2025_2401.15719,
  title={ Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning },
  author={ R. Srikant },
  journal={arXiv preprint arXiv:2401.15719},
  year={ 2025 }
}