It is a common goal of image processing to obtain the highest quality image possible from a given set of input data. Image quality is typically judged in terms of “resolution,” which generally refers to the accuracy and precision of the data which formed the image, and can include spatial resolution (e.g., pixels per unit area or per image), the precision and accuracy of the characteristics of each pixel (e.g., dynamic range of brightness and/or color), and/or the number of image frames per unit time (e.g., for video applications). Conventional image-processing algorithms typically ignore information and knowledge which is extrinsic to the data being processed, but is nonetheless relevant. As a result, the resolution of an image produced using a conventional image-processing algorithm is reduced. In particular, although input image can theoretically have virtually any characteristics, the characteristics of most images are limited by the laws of physics governing the physical objects being imaged. In addition, although an object can in theory have almost any shape, there are certain shapes which tend to recur more frequently than others.
For example, although it is theoretically possible for an image to include a brightness impulse—i.e., an infinitely tiny point of infinite brightness—such a phenomenon never occurs in real life. In addition, images of real objects are often filled with simple geometric shapes such as, for example, lines, rectangles, ellipses, flat surfaces, and smoothly rounded surfaces. Rarely in practice does an image of a real, physical scene consist primarily of a set of completely random shapes distributed in a completely random fashion. Yet, conventional image-processing algorithms largely ignore the aforementioned practical limitations which govern most images.
One conventional approach for enhancing dynamic range is to sequentially capture multiple images of the same scene using different exposure amounts. The exposure amount for each image is controlled by varying either the aperture of the imaging optics or the exposure time of the image detector. A high-exposure image (i.e., an image captured using a wide aperture or long exposure time) will tend to be saturated in the bright scene areas but more accurately captured in the dark regions. In contrast, a low-exposure image (i.e., an image captured using a small aperture or short exposure time) will tend to have less saturation in bright regions, but may be too dark and noisy in dark areas. The complementary nature of such high-exposure and low-exposure images allows them to be combined into a single high dynamic range image. Such an approach can be further enhanced by using the acquired images to compute the radiometric response function of the imaging system.
The above methods are better suited to static scenes than to moving scenes, because in order to obtain good results, it is preferable for the imaging system, the scene objects, and the radiances of the objects to remain constant during the sequential capture of images under different exposures. However, the stationary scene requirement has, in some cases, been remedied by the use of multiple imaging systems. In such an approach, beam splitters are used to generate multiple copies of the optical image of the scene. Each copy is detected by an image detector whose exposure is preset by using an optical attenuator or by setting the exposure time of the detector to a particular value. The exposure amount of each detector is set to a different value. This approach has the advantage of producing high dynamic range images in real time. Real time imaging allows the scene objects and/or the imaging system to move during the capture, without interfering with the processing of the multiple image copies. A disadvantage is that this approach is expensive because it requires multiple image detectors, precision optics for the alignment of all of the acquired images, and additional hardware for the capture and processing of the multiple images.
Another approach which has been used for high dynamic range imaging employs a special CCD design. In this approach, each detector cell includes two sensing elements having potential wells of different sizes—and therefore, different sensitivities. When the detector is exposed to the scene, two measurements are made within each cell and the measurements are combined on-chip before the image is read out. However, this technique is expensive because it requires fabrication of a complicated CCD image sensor. In addition, the spatial resolution of the resulting image is reduced by a factor of two, because the two sensing elements occupy the same amount of space as two pixels would occupy in an image detector with single sensing element cells. Furthermore, the technique requires additional on-chip electronics in order to combine the outputs of the two sensing elements in each detector cell before sending the signals off the chip for further processing.
An “adaptive pixel” approach for high dynamic range imaging has also been proposed. Such an approach employs a solid state image sensor in which each pixel includes a computational element which measures the time required to accumulate charge in the potential well to full capacity. Because the well capacity is the same for all pixels, the time required to fill a well is proportional to the intensity of the light incident on the corresponding pixel. The recorded time values are read out and converted to a high dynamic range image. In some cases, this approach can provide increased dynamic range. However, although a 32×32 cell device has been implemented, it is likely to be difficult to scale the technology to high resolution without incurring high fabrication costs. In addition, because exposure times tend to be large in dark scene regions, such a technique is likely to have relatively high susceptibility to motion blur.
Image data can also be captured in the form of polarization data associated with image pixels. In particular, it has been proposed that an individual polarization filter be placed in front of each element in a detector array. For example, individual polarization filters can be added to individual color filters covering an array of detectors. The outputs of the detectors beneath the polarization filters can be used to estimate the polarization of the light striking adjacent detectors. However, such a technique sacrifices spacial resolution because with respect to the polarization data, it treats the entire region surrounding a pixel as a single pixel.