ONLINE ESTIMATION OF HIDDEN MARKOV MODEL PARAMETERS BASED ON THE KULLBACK-LEIBLER INFORMATION MEASURE

In this paper, sequential or ''on-line'' hidden Markov model (HMM) signal processing schemes are derived and their performance illustrated in simulation studies. The on-line algorithms are sequential expectation maximization (EM) schemes and are derived by using stochastic approximations to maximize the Kullback-Leibler information measure. The schemes can be implemented either as filters or fixed-lag or sawtooth-lag smoothers. They yield estimates of the HMM parameters including transition probabilities, Markov state levels, and noise variance. In contrast to the off-line EM algorithm (Baum Welch scheme) which uses the fixed-interval ''forward-backward'' scheme, the on-line schemes have significantly reduced memory requirements, improved convergence (as shown in simulations), and can estimate HMM parameters that vary slowly with time or undergo infrequent jump changes. Using similar techniques we also derive on-line schemes to extract finite-state Markov chains imbedded in a mixture of white Gaussian noise (WGN) and deterministic signals of known functional form with unknown parameters. In particular, deterministic periodic signals with unknown and time-varying amplitudes and phases are considered. Simulations presented show that these schemes satisfactorily estimate the HMM parameters and also the time-varying amplitudes and phases.