Abstract
Recently, facial expression recognition has attracted much attention due to its potential for applications in the area of human computer interaction. This thesis investigates the effects of features and pose on facial expressions recognition. Several different features are evaluated to investigate how temporal, orientation and multi scale information effects facial expression recognition accuracy. Also in this thesis, the effects of pose on facial expression recognition is investigated by classifying expressions over a range of poses from frontal to profile view. An efficient edge based approach which builds strong classifiers from boosting is presented. Small coherent edge fragments are extracted from the area in and around the face. A classifier bank is then assembled from candidate edge fragments from all the training examples. Boosting is used to choose an optimal subset of features from the classifier bank to form a strong discriminatory classifier. The final boosted classifier provides a binary decision for object recognition. An investigation of different fusion methodologies for a multi-class ensemble is also presented. This approach is extended into the temporal domain using a temporalboost algorithm which allows weak classifier to incorporate previous frames responses when evaluating the current weak classifier. Research into facial expression recognition has predominantly been applied to face images at frontal view only. Some attempts have been made to produce pose invariant facial expression classifiers. However, most of these attempts have only considered yaw variations of up to 45 , where all of the face is visible. Little work has been carried out to investigate the intrinsic potential of different poses for facial expression recognition. A sequential 2 stage approach is taken for pose classification and view dependent facial expression classification to investigate the effects of yaw variations from frontal to profile views. Recent databases, BU3DFE and multi-pie, allows empirical investigation of facial expression recognition for different viewing angles. Local binary patterns (LBPs) and variations of LBPs as texture descriptors are investigated. Such features allow investigation of the influence of orientation and multi-resolution analysis for multiview facial expression recognition. The main contributions of the work presented in this thesis are the following: fast efficient edge features for static facial expression recognition are introduced. This approach is extended into the temporal domain using temporalboost. Variations of LBP features are compared and contrasted to investigate the influence of orientation and multi-scale information on recognition results. A novel feature for facial expression recognition is introduced Local Gabor Binary Patterns. The effects of pose on facial expression recognition are also investigated on the BU3DFE and multi-pie databases. Experiments analyse how pose effects overall expression recognition as well as individual expression recognition.