Abstract
This paper presents an algorithm for accurately aligning two images of the same scene captured simultaneously by sensors operating in different wavebands (e.g. TV and IR). Such a setup is common in image fusion systems where the sensors are physically aligned as closely as possible and yet significant image mis-alignment remains due to differences in field of view, lens distortion and other camera characteristics. Our proposed registration method involves numerically minimising a global objective function defined in terms of local normalised correlation measures. The algorithm is demonstrated on real multimodal imagery and applications to imagefusion are considered. In particular we illustrate thatfused image quality is closely related to the degree ofregistration accuracy achieved. To maintain this accuracy in real systems it is often necessary to continuously update the transform over time. Thus we extend our registration approach to execute in real time on live imagery, providing optimal fused imagery in the presence ofrelative sensor motion andparallax effects.