A Unified Framework for Risk-sensitive Markov Decision Processes with Finite State and Action Spaces

We invent a unified framework to incorporate risk into the Markov decision processes (MDPs), via prospect maps, which generalize the idea of coherent/convex risk measures in mathematical finance. Most of the existing risk-sensitive approaches in various literature concerning with decision-making problems are covered by the framework as special instances. Within the framework, we solve the optimal control problems with respect to two criteria, the newly invented temporal discounted criterion, which generalizes the conventional discount scheme, and the average criterion, by value iteration algorithms under different assumptions. Two online algorithms are proposed to solve the optimal controls problem for real applications.