Multi-biometrics using facial appearance, shape and temperature

We present results of the first study to examine individual and multi-modal face recognition using 2D, 3D and infrared images of the same set of subjects. Each sensor captures different aspects of human facial features; appearance in intensity representing surface reflectance from a light source, shape data representing depth values from the camera, and the pattern of heat emitted, respectively. We employ a database containing a gallery set of 127 images and an accumulated time-lapse probe set of 297 images. Using a PCA-based approach tuned separately for 2D, 3D and IR, we find rank-one recognition rates of 90.6% for 2D, 91.9% for 3D and 71.0% for IR. Combining each pair of modalities, we find a multi-modal rank-one recognition rate of 98.7% for 2D-3D, 96.6% for 2D-IR and 98.0% for 3D-IR. When all three modalities are combined, we obtain 100% recognition. The results shown in this study appear to support the conclusion that the path to higher accuracy and robustness in biometrics involves use of multiple biometrics rather than the best possible sensor and algorithm for a single biometric.