Constant-Memory Strategies in Stochastic Games: Best Responses and Equilibria

(Here is a short version, see our paper for the complete abstract.)In this work, we comprehensively investigate the concept of constant-memory strategies in stochastic games. We first establish some results on best responses and Nash equilibria for behavioral constant-memory strategies, followed by a discussion on the computational hardness of best responding to mixed constant-memory strategies. Those theoretic insights later empower a generative framework for studying generalizability of single-agent RL algorithms.

@article{zhu2025_2505.07008,
  title={ Constant-Memory Strategies in Stochastic Games: Best Responses and Equilibria },
  author={ Fengming Zhu and Fangzhen Lin },
  journal={arXiv preprint arXiv:2505.07008},
  year={ 2025 }
}