OPTIMAL INFINITE-HORIZON UNDISCOUNTED CONTROL OF FINITE PROBABILISTIC SYSTEMS

A finite-point, finite-state, finite-output stochastic control problem with imperfect state observation and-classical information pattern is shown to be meaningful as the horizon increases without bound and the discount rate approaches unity. The plant model, a finite probabilistic system, includes the Markov decision and partially-observed Markov decision problems as special cases. Under conditions resembling controllability and observability in linear systems it is shown that: an optimal strategy exists, it may be realized by a stationary policy on the state estimate, its performance does not depend on the initial state distribution, and convergence rates for its finite-horizon and discounted performances are readily established.