Abstract
Realistic 3D computer models of moving human faces has been a longstanding problem in computer graphics and vision research. This thesis addresses the problem of capturing, modelling and analysing 3D facial dynamics and provides a framework for the realistic synthesis of face sequences. Currently, 3D face reconstruction methods cannot provide dynamic shape and appearance information at full video rate. The contribution of this work is the development of a novel multi-view 3D capture system which simultaneously acquires dynamic face shape and appearance at full video rate. Optical filters are used to acquire shape using active stereo in the infra-red zone and colour appearance in the visible zone of the spectrum. The captured shape and colour appearance of the face is mapped onto an ellipsoidal primitive introducing an efficient 3D video representation. This representation forms the basis for spatio-temporal processing to extract accurate non-rigid deformation during speech and expressions. Results and evaluation suggest that dynamic 3D faces can be reconstructed with a quality comparable to captured video. An audio-driven visual speech synthesis application is also presented which demonstrates that realistic synthesis of 3D face sequences is feasible using the observed dynamics.