Abstract
Mobile camera platforms are becoming omni-present, spawning many new applications. This trend is accelerated by cheap, small and powerful cameras. Although the cameras in these mobile devices are already of good quality, they still have physical limitations in terms of spatial resolution, dynamic range and temporal information. In this thesis we investigate algorithmic solutions using image fusion to overcome those limitations. We consider three main techniques: super-resolution, high-dynamic-range imaging and motion detection. The first one aims at generating high-resolution images from low-resolution input videos. The second one combines multiple images taken with different camera settings to generate an image with a greater dynamic range than in any of the input images. The third technique analyzes multiple consecutive frames in a video to extract pixels belonging to moving objects. As mobile cameras are not static, they impose the challenge of compensating the ego-motion. Therefore, we investigate for each of the image fusion approaches the required image registration steps and propose the best suited algorithms. For the application of increasing resolution, we present a solution to enhance low-resolution videos using multi-frame reconstruction-based superresolution. We propose to use high-resolution images from the Internet as priors within a maximum-a-posteriori formulation. We demonstrate that this superresolution framework increases the resolution of low-quality input videos taken with mobile cameras. Similar to superresolution, high-dynamic-range imaging is also an algorithmic solution to overcome the physical limitation of a sensor. Many approaches have been proposed for both enhancement applications individually, but little research has been carried out to provide solutions which address both problems simultaneously. We present an approach that combines multi-frame reconstruction-based superresolution and a new minimal high-dynamic-range imaging method into a unified framework. Relating multiple consecutive frames from a video allows to detect moving objects. This task requires many registration operations, hence demanding very efficient registration methods. We present a fast algorithm to compute homographies based on phase correlation, which benefits from implementations using GPUs. Despite accurate registration, errors in the motion extraction process due to parallax and sensor noise are inevitable and generate false-alarms. We present an efficient filtering scheme, which significantly reduces the false detections, thus improves the performance of moving object detection. Many image enhancement algorithms that increase the level of image details are motivated by the hope that they improve the performance of subsequent computer vision tasks. We investigate how much the performance of two common applications, i.e. image retrieval and scene recognition, can be increased when applying image filters as a pre-processing step. We consider standard and advanced gradient-based filtering techniques using state-of-the-art benchmark datasets for evaluation and show that reducing the level of image details, e.g. in terms of image abstraction, improves retrieval and recognition.