A note on the article "On Exploiting Spectral Properties for Solving MDP with Large State Space"

We improve a theoretical result of the article "On Exploiting Spectral Properties for Solving MDP with Large State Space" showing that their algorithm, which was proved to converge under some unrealistic assumptions, is actually guaranteed to converge always.