Maximum Mutual Information Neural Networks for Hybrid Connectionist-HMM Speech Recognition Systems

This paper proposes a novel approach for a hybrid connectionist-hidden Markov model (HMM) speech recognition system based on the use of a neural network as vector quantizer. The neural network is trained with a new learning algorithm offering the following innovations: 1) It is an unsupervised learning algorithm for perceptron-like neural networks that are usually trained in supervised mode. 2) Information theory principles are used as learning criteria, making the network especially suitable for combination with a HMM-based speech recognition system. 3) The neural network is not trained using the standard error-backpropagation algorithm but using instead a newly developed self-organizing learning approach. The use of the hybrid system with the neural vector quantizer results in a 25% error reduction compared with the same HMM system using a standard k-means vector quantizer. The training algorithm can be further refined by using a combination of unsupervised and supervised learning algorithms. Finally, it is demonstrated how the new learning approach can be applied to multiple-feature hybrid speech recognition systems, using a joint information theory-based optimization procedure for the multiple neural codebooks, resulting in a 30% error reduction.