Separability of pose and expression in facial tracking and animation

We explore the application of facial tracking to automated re-animation. To this end; it is necessary to recover both head-pose and facial expression from the facial movement of a performer. However, both effects are coupled. This is a serious problem, which previous studies haven't fully considered. The solution to this interaction problem proposed here is to solve explicitly, at-each timestep, for pose and expression variables. In principle this is a nonlinear inverse problem. However, appropriate parameterisation of pose in terms of affine transformations with parallax, and of expression in terms of key-frames, reduces the problem to a bilinear one. This can then be solved directly by Singular Value Decomposition. Thus actor-driven animation has been implemented in real-time, at video field-rate, using two Indy desktop workstations.