Abstract
A fundamental goal of computer vision is the ability to detect and track semantic objects. This can benefit many applications ranging from video compression to video content analysis. Although humans can easily identify changes in meaningful objects in a video sequence, automatic detection and identification of semantic video objects is still a difficult task for current computer vision research due to challenges including cluttered backgrounds, arbitrary non-rigid motion, changing appearance patterns of non-rigid objects, and partial or full object occlusions. The key to address these challenges is to improve current computer vision systems with the flexibility to handle new input. Recent developments in 3D data acquisition technologies facilitate the provision of different types of depth information for objects and video scenes. Thus, this thesis aims to address the development of techniques for better utilization of 3D scene data in semantic object segmentation and tracking. The video object detection problem is addressed by means of a depth and motion based video object segmentation approach. The combination of depth map segmentation and motion detection forms the foreground object masks. A depth assisted multiple objects tracking system is developed based on these object masks. The proposed tracking algorithm utilizes a stereo-vision system's ability of separating occluding objects at different depth layers where different tracking strategies are employed according to the various occlusion situations and object depth layers. To further improve the performance of video object detection and identification, this thesis explores the issues of 3D objects and scene model reconstruction, which can reveal the geometric relationship between moving objects. The proposed approach combines the advantages of Delaunay triangulation method and the region growing method to achieve efficient surface reconstruction. Furthermore, a fuzzy clustering based 3D object decomposition algorithm is also discussed. Decomposing objects into meaningful components recovers useful structural properties of a 3D object. Thus, it is an important step for object recognition and video content analysis.