Frame rate and viseme analysis for multimedia applications to assist speechreading

Current video conference and phone systems do not provide the necessary temporal resolution and motion for speechreading. In this paper the perceptual boundaries which effect speechreading performance are investigated. Analysis of the relationships between viseme groupings, accuracy of viseme recognition and presentation frame rate is presented based on the results of subject testing. Results reveal a minimum frame rate of 10 frames per second (fps) for distinguishing viseme groupings. Confusion analysis results demonstrate the importance of the tongue and teeth oral features for speechreading. These results are critical to the design of speech-assisted video systems to enhance speechreading for individuals with impaired hearing.