Image Stabilization and Enhancement
Various computer vision techniques are successfully used in many different applications like video compression, target tracking, or object recognition. In particular, digital image stabilization plays a key role in contemporary digital video systems, such as digital cameras and navigation tools.
During video capture, the camera often vibrates, which leads to undesirable trembling of the recorded video. In the simplest case, this phenomenon only involves a small vibration of the camera's optic axis. However, in principle, large vibrations (e.g. up to half a frame) are also possible, especially if strong magnification is used during the recording. More complicated cases might also develop. For example, the camera might not only vibrate, but also change its orientation and position in space. In any case, the problem of compensating the movement of the camera in order to obtain a stabilized image arises.
The objective is to stabilize and enhance images obtained from a limited-resolution imaging sensor (video/infrared) on a moving platform. The developed Image Stabilization & Enhancement System is based on the adaptive spatial-temporal processing that is combined with an advanced jitter estimation/compensation/stabilization algorithm. Implementation of a new approach to Video/Infrared Image Stabilization Technology in software results in sub-pixel translational stabilization (up to 1-3 percent of a pixel), accurate rotational stabilization (3 axis system – roll, pitch and yaw), scaling (compensation for zooming), and resolution enhancement (double /triple the resolution).
Wavelet and Curvelet features extraction and matching – Wavelet and curvelet features provide robust performance for scenes with intense background clutter. Computational complexity is reduced tremendously compared to Fourier feature analysis. The multi-resolution and orthogonality properties of the wavelet bases allow for on-the-fly selection of subbands in the image data.
Spatial-temporal image processing – The spatial-temporal image estimation technique is based on a multi-parametric approximation of the image, which after estimation of parameters, leads to an adaptive spatial-temporal filter. The filter coefficients are adaptively calculated according to the minimum distance algorithm. The adaptive spatial-temporal filter allows very accurate image estimation, regardless of its spatial variation. The filter not only whitens the residuals but also corrects all translational, rotational, and parallax distortions. This results in sub-pixel registration, taking advantage of small random sensor vibrations to actually increase the image resolution (super-stabilization and super-resolution).
UAV applications – The developed technology is being implemented for Unmanned Arial Vehicle (UAV) surveillance applications supported by the NAVY.
CAMS investigators include Alexander Tartakovsky, Rudolf Blazek, and Prashant Pradeep.
Experimental Results:
These two movies illustrate the performance of the Image Stabilization & Enhancement System. Movie 1 contains extremely severe rotation that is completely compensated by the system. Movie 2 includes rotation, zooming and translations.
Most Recent Results
ImageStabilizer v1.0 is a software framework aimed at solving the shift-rotation stabilization problem as well as zooming. Along with the motion recovery algorithms developed at CAMS, ImageStabilizer v1.0 creates an efficient video stabilizing tool, successfully securing the following three features:
· Robustness against noise, regardless of its spatio-temporal characteristics;
· Real-time performance on modern equipment;
· Sub-pixel accuracy in estimating the transformations between frames (e.g. shifts, rotations and zooming).
Real-time performance is achieved first, through using Microsoft Direct X technology for rendering images and the Intel Integrated Performance Primitives (IPP) along with Intel OpenCV for optimization. And secondly, by the appropriate design of the stabilization algorithm, where the so-called “feature-points” representing the most informative video fragments are used to estimate the inter frame transformations by predicting the future position thereof based on the history.
The software package includes a Graphical User Interface (GUI) that allows for displaying the original video stream, stabilized video stream and the mosaic video. The GUI is shown in Figure 1. Sample mosaics are shown in Figures 2 and 3.
Fig. 1 Graphical User Interface: General View.
Fig. 2 Sample Mosaic #1
Fig. 3 Sample Mosaic #2