Depth Camera-Based Facial Expression Recognition System Using Multilayer Scheme

The analysis of a facial expression in telemedicine and healthcare plays a significant role in providing sufficient information about patients such as stroke and cardiac in monitoring their expressions for better management of their diseases. Facial expression recognition (FER) improves the level of interaction between human-to-human communications. The human face has a major contribution for such types of communications, which consists of lips, eyes and forehead that are considered the most informative features for FER. There are some parameters that make FER a challenging task that includes high similarity among different expressions that makes it difficult to distinguish these expressions with a high accuracy. Moreover, most of the previous works used existing available standard datasets and all the datasets were pose-based datasets, and they have some privacy issues because of utilizing video (RGB) cameras. Accordingly, this work presents a multilayer scheme for FER to handle these issues. In the proposed FER system, we utilized a depth camera in order to solve the privacy concerns, and the accuracy of this camera is not affected by any kind of environmental parameters. Similarly, the depth camera automatically detects and extracts the faces based on the distance between the camera and subject. For global and local feature extraction, principal component analysis (PCA) and independent component analysis (ICA) were used. A hierarchical classifier was used, where the expression category was recognized at the first level, followed by the actual expression recognition at the second level. For the entire experiments, an n-fold cross-validation scheme (based on subjects) was employed. The proposed FER system achieved a significant improvement in accuracy (98.0%) against the existing methods.