OPTIMALITY OF STRUCTURED POLICIES IN COUNTABLE STAGE DECISION PROCESSES

Multistage decision processes are considered, in notation which is an outgrowth of that introduced by Denardo. Certain Markov decision processes, stochastic games, and ris-sensitive Markov decision processes can be formulated in this notation. Conditions are identified which are sufficient to prove that, in infinite horizon nonstationary processes, the optimal infinite horizon (present) value exists, is uniquely defined, is that is called ″structured,″ and can be found by solving Bellman's optimality equations; epsilon -optimal strategies exist; an optimal strategy can be found by applying Bellman's optimality criterion; and a specially identified kind of policy, called a ″structured″ policy is optimal in each stage.