ALPHA-NETS - A RECURRENT NEURAL NETWORK ARCHITECTURE WITH A HIDDEN MARKOV MODEL INTERPRETATION

A hidden Markov model isolated word recogniser using full likelihood scoring for each word model can be treated as a recurrent 'neural' network. The units in the recurrent loop are linear, but the observations enter the loop via a multiplication. Training can use back-propagation of partial derivatives to hill-climb on a measure of discriminability between words. The back-propagation has exactly the same form as the backward pass of the Baum-Welch (EM) algorithm for maximum-likelihood HMM training. The use of a particular error criterion based on relative entropy (equivalent to the so-called Mutual Information criterion which has been used for discriminative training of HMMs) can have derivatives which are interestingly related to the Baum-Welch re-estimates and to Corrective Training. © 1990.