Facial motion analysis for content-based video coding

Automatic wire-frame fitting and automatic wire-frame tracking are the two most important and most difficult issues associated with semantic-based moving image coding. A novel approach to high speed tracking of important facial features is presented as a part of a complete fitting-tracking system. The method allows real-time processing of head-and-shoulders sequences using software tools only. The algorithm is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. Each important facial feature (the left eye, the right eye, the nose and the lips) is tracked separately using the same method. The algorithm was tested on widely used head-and-shoulders video sequences containing the speaker's head pan, rotation and zoom with remarkably good results. These experiments prove that it is possible to maintain tracking even when the facial features are partially occluded. (C) 2000 Academic Press.