View i2k Align View i2k Retina
i2k Align Technology


Given two images, that may or may not overlap, i2Align™ has two inter-related jobs: to decide if the images do indeed overlap and, if so, to compute an accurate mapping between the two images. i2Align™ achieves this using a three-step process: (1) generating initial estimates of the inter-image transformation, (2) refining the most promising initial estimates, and (3) deciding if any of the resulting estimates align the same structures from the two images. The earlier Dual-Bootstrap algorithm (GDB-ICP), IEEE Trans. on Pattern Analysis and Machine Intelligence Nov 2007, which we developed with colleagues and fellow students at Rensselaer Polytechnic Institute, used matching of SIFT features to generate initial estimates, a region growth and model selection algorithm to refine the estimates, and a combination of stability, accuracy and orientation-consistency measures to make the final decision. Each of these components has been replaced with a proprietary new technique in i2Align™, which is now more intelligent and faster than the Dual-Bootstrap algorithm.

The power of the algorithm allows it to produce inter-image alignments such as shown in the following gif animation.

Transformation Models

i2k Align™ includes similarity, affine, homography, planar (homography plus radial-lens distortion), cylindrical (also with radial-lens distortion) and quadratic transformation models. See xforms-and-matched.pdf. These are accurate enough to produce well-aligned images in a wide variety of situations. When local distortions and parallax are present in the images, the images will still usually be aligned, with the distortions and parallax appearing as inter-image shifts.

The animation below shows three image aligned iPhone images taken at a resort in Hawaii.  The misalignments due to a small change in viewpoint (and individuals walking in the field of view) may be seen in the trees and, especially, in the bushes on the front right.

Even in this case, the resulting correspondences are generally quite accurate, and for montaging, our seam-selection algorithm can remove most of the motion artifacts, as seen in the following montage

Geometric Distortion Correction

When multiple images are aligned, there are extra degrees of freedom in the transformations, adding flexibility to the mapping of the images. i2k Align™ uses this flexibility to undo as much of the mapping distortion as it can, substantially improving the overall appearance of the montage. The following two montages show the difference between not using distortion correction (top) and using distortion correction (bottom). Particular attention should be paid to the horizon.

Automatic Masking

Due to the physical set-up of the image acquisition process, some non-photographic images contain an outer region that is outside the true image content. Automatically cropping this area is needed for clean montages, and it aids in the align process, especially for low overlap images. When this exterior region is all black or all white, the masking process is trivial, but it can become considerably harder for poor quality images (e.g. images scanned from slides). i2k Align™ offers the option to automatically compute a convex mask for images labeled "Other Modalities".

Uneven Illumination and Vignetting Correction

Some photographic lenses and and most retinal fundus cameras produce images that tend to be darker on the image periphery than near the image center. For photographic lenses this is due to "vignetting", while for fundus images it is caused by uneven illumination.  In either case, good quality montages and better overall appearance may be obtained if this can be automatically corrected.

DualAlign™ has developed new algorithms for automatically correcting vignetting and uneven illumination both in a single image and across multiple images. These are applied in the photographic montaging process of i2k Align™.

Camera Parameter Estimation and Illumination Correction

Different settings on the same camera can produce different recorded colors and overall image intensities. Even with the same aperture and shutter-speed settings, internal correction algorithms inside digital cameras can produce different color values for the same object in two different images. This problem is particularly difficult for cell-phone cameras, which include essentially no user control. DualAlign™ has developed algorithms for camera parameter estimation, brightness correction and, to some degree, white balancing (color correction) that eliminate many of these problems. 

As an example, the montage below combines four images taken of the Christian Science Center using the camera on a Blackberry Curve. Notice the color transitions in the montage, especially between the bottom and top of the building on the right and in the sky.

Now look at what happens after the i2k Align™ camera estimation, brightness correction and white-balancing algorithms are applied:

Seam Selection

Photographers are limited in the panoramas they can build if they need to use a tripod and require a stationary scene. On the other hand, using a hand-held camera and photographing a dynamic scene requires software that eliminates artifacts due to parallax and corrects for movement in the scene. i2k Align™ includes algorithms that select seams and blending regions between images based on a detailed model of the image misalignments and color differences that remain after illumination correction.We have already seen an example above where the misalignments in three images taken at a resort in Hawaii were removed to form a seamless montage. This is not limited to photographic images. The montage of six thermal images below has had the image parallax removed through seam selection.

Six Input Images The Resulting Montage