Automatic facial expression recognition using facial animation parameters and multistream HMMs

被引:133
作者
Aleksic, Petar S. [1 ]
Katsaggelos, Aggelos K. [1 ]
机构
[1] Northwestern Univ, Dept Elect Engn & Comp Sci, Evanston, IL 60208 USA
关键词
facial expression recognition; multistream HMMs; facial animation parameters;
D O I
10.1109/TIFS.2005.863510
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
The performance of an automatic facial expression recognition system can be significantly improved by modeling the reliability of different streams of facial expression information utilizing multistream hidden Markov models (HMMs). In this paper, we present an automatic multistream HMM facial expression recognition system and analyze its performance. The proposed system utilizes facial animation parameters (FAPs), supported by the MPEG-4 standard, as features for facial expression classification. Specifically, the FAPs describing the movement of the outer-lip contours and eyebrows are used as observations. Experiments are first performed employing single-stream HMMs under several different scenarios, utilizing outer-lip and eyebrow FAPs individually and jointly. A multistream HMM approach is proposed for introducing facial expression and FAP group dependent stream reliability weights. The stream weights are determined based on the facial expression recognition results obtained when FAP streams are utilized individually. The proposed multistream HMM facial expression system, which utilizes stream reliability weights, achieves relative reduction of the facial expression recognition error of 44% compared to the single-stream HMM system.
引用
收藏
页码:3 / 11
页数:9
相关论文
共 39 条
[21]  
MASE K, 1991, IEICE TRANS COMMUN, V74, P3474
[22]  
OKAWA S, 1999, P 6 EUR C SPEECH COM, V2, P603
[23]  
OLIVER N, 1997, P CVPR97
[24]   Spotting segments displaying facial expression from image sequences using HMM [J].
Otsuka, T ;
Ohya, J .
AUTOMATIC FACE AND GESTURE RECOGNITION - THIRD IEEE INTERNATIONAL CONFERENCE PROCEEDINGS, 1998, :442-447
[25]  
Padgett C, 1997, ADV NEUR IN, V9, P894
[26]  
PADGETT C, 1995, P 2 JOINT S NEUR COM, V5, P91
[27]  
Pandzic I. S., 2002, MPEG4 FACIAL ANIMATI
[28]   Automatic analysis of facial expressions: The state of the art [J].
Pantic, M ;
Rothkrantz, LJM .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2000, 22 (12) :1424-1445
[29]  
Pardàs M, 2002, INT CONF ACOUST SPEE, P3624
[30]   Facial animation parameters extraction and expression recognition using Hidden Markov Models [J].
Pardàs, M ;
Bonafonte, A .
SIGNAL PROCESSING-IMAGE COMMUNICATION, 2002, 17 (09) :675-688