STATE-OF-THE-ART - A SURVEY OF PARTIALLY OBSERVABLE MARKOV DECISION-PROCESSES - THEORY, MODELS, AND ALGORITHMS

This study surveys models and algorithms dealing with partially observable Markov decision processes. A partially observable Markov decision process (POMDP)is a generalization of a Markov decision process which permits uncertainty regarding the state of a Markov process and allows for state information acquisition. A general framework for finite state and action POMDP's is presented. There is also a brief discussion of the development of POMDP's and their relationship with other decision processes. A wide range of models in such areas as quality control, machine maintenance, internal auditing, learning, and optimal stopping are discussed within the POMDP-framework. Algorithms for computing optimal solutions to POMDP's are presented.