Soldiers often find themselves in situations in which there is reduced or no visibility of the battlefield, especially at night. There is a need for providing visual information to soldiers in nighttime environments and day or night environments with visual obstructions such as fog or smoke. In such situations, the soldier may be equipped with soldier-worn, hand-held, and/or vehicle-based night vision and enhanced-vision systems. These systems enhance visibility in the visible range of frequencies of electromagnetic radiation, or provide “sight” in the infrared range of frequencies.
An improvement over individual systems that provide visibility over a single range of frequencies of electromagnetic radiation or combine visibility in several ranges of frequencies with different pieces of equipment is to combine the video of a long wave infrared (LWIR) source, a short wave infrared (SWIR) source and a standard visible source, into a single image using a single piece of equipment, thereby providing significantly enhanced visibility of the scene. Another example is combining video from two cameras that point at the same scene, but with different focal length, providing enhanced depth of focus. A third example is combining video from two cameras that have a different aperture setting, providing significantly enhanced dynamic range to the display. In all these applications, it is desirable to preserve the most significant details from each of the video streams on a pixel-by-pixel basis. Such systems employ a technique known in the art as image fusion.
One image fusion technique known in the art is to perform an averaging function of the multiple video streams. However, contrast is reduced significantly and sometimes detail from one stream may cancel detail from another stream. Laplacian pyramid fusion on the other hand provides excellent automatic selection of the important image detail for every pixel from multiple images at multiple image resolutions. By performing selection in the multi-resolution representation, the reconstructed—fused—image provides a more natural-looking scene. In addition, the Laplacian pyramid fusion algorithm allows for additional enhancement of the video. It may provide multi-frequency sharpening, contrast enhancement, and selective de-emphasis of image detail in either video source.
However, current multi-scale, feature-selective fusion techniques employing Laplacian pyramid decomposition/construction do not work well on high dynamic range (HDR), high noise imagery. Performing dynamic range adjustment on the input images before fusion may ameliorate some of these problems. Various techniques such as histogramming and linear stretching have been introduced to take better advantage of the input image dynamic range. But these techniques still do not adequately deal with localized areas of low contrast and do not address issues associated with noise in the input images.
Accordingly, what would be desirable, but has not yet been provided, is a method and system for effectively and automatically fusing images from multiple cameras of different frequency bands (modalities) that benefit from the advantages of Laplacian pyramid decomposition/construction while being immune to low contrast and the presence of noise.