1. Field of the Invention
The present invention relates to cleaning methods for foggy images, and more particularly to a cleaning method for foggy images based on atmospheric scattering theory and color analysis.
2. Description of the Prior Art
In recent years, economic growth has spurred an increased interest in transportation issues. As roadway monitoring systems adopt computer visualization and image processing techniques more widely, traffic volume is steadily increasing, leading to research and development of all types of transportation-related technologies. Among a great volume of traffic-monitoring research, an Intelligent Transportation System (ITS) is one method utilized to solve traffic problems. ITS incorporates communications, control, electronics, and information technologies to put limited traffic data to its greatest use, so as to improve transportation safety, improve quality of life, and enhance economic competitiveness. ITS technologies include microelectronics technology, automated artificial intelligence, sensor technology, communications systems, and control. One of the most important ITS technologies is computer visualization. Because ITS' effective operation relies on accurate, real-time traffic monitoring parameters, not only does application of image processing and computer visualization techniques lower cost (by greatly reducing labor costs) and also make for easy installation, image processing and computer visualization techniques also provide measurement and monitoring of larger areas, so as to obtain more diverse information. In addition to capturing traditional traffic parameters, such as traffic volume and speed, ITS further detects traffic jams and traffic causes, or anticipates traffic accidents.
However, due to the influence of foggy conditions brought about by seasonal weather, because visibility is reduced when computer visualization is utilized for traffic monitoring, images captured may be unclear, making the images unrecognizable, and keeping the computer visualization application from being able to obtain accurate color information during processing. Meteorology teaches that the condensation processes of fog and clouds are similar, both occurring through cooling. However, the reasons for cooling are different in fog and clouds. Fog has many types, and forms for different reasons. The physical processes and conditions for forming fog are also very complex. Fog on a coast or on an ocean is typically advection fog, whereas fog on land or on a mountain is typically radiation fog.
Fog is one of the most influential types of weather in transportation and everyday life. Fog affects visibility in the surrounding environment, obstructing traffic control systems. For example, air traffic control towers, road traffic monitors, and harbor ship controllers are all affected by fog, which increases danger. In addition to traffic control, equipment that relies on the aid of monitoring systems is also affected, which greatly reduces the value of image monitoring systems.
Currently, many authors have introduced foggy image restoration methods, including at least the following:    [1] Srinivasa G. Narasimhan and Shree K. Nayar, “Vision and the Atmosphere,” International Journal of Computer Vision, Vol. 48, No. 3, pp. 233-254, August 2002.    [2] Srinivasa G. Narasimhan and Shree K. Nayar, “Contrast Restoration of Weather Degraded Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 6, pp. 713-724, 2003.    [3] Srinivasa G. Narasimhan and Shree K. Nayar, “Interactive (De) Weathering of an Image using Physical Models,” Proceedings of the ICCV workshop on Color and Photometric Methods in Computer Vision, pp. 1387-1394, 2003.    [4] Robby T. Tan, Niklas Pettersson and Lars Petersson, “Visibility Enhancement for Roads with Foggy or Hazy Scenes,” IEEE Intelligent Vehicles Symposium, pp. 19-24, June 2007.    [5] Robby T. Tan, “Visibility in Bad Weather from a Single Image”, IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, June 2008.    [6] Didier Aubert and Nicolas Hautiere, “Contrast Restoration of Foggy Images through use of an Onboard Camera,” IEEE Conference on Intelligent Transportation Systems, pp. 601-606, September 2005.    [7] Hiroshi Murase, Kenji Mori, Ichiro Ide, Tomokazu Takahashi, Takayuki Miyahara and Yukimasa Tamatsu, “Recognition of foggy conditions by in-vehicle camera and millimeter wave radar”, 2007 IEEE Intelligent Vehicles Symposium, Vol. 3, No. 5, pp. 87-92, June 2007.    [8] Yoav Y. Schechner, Srinivasa G. Narasimhan, and Shree K. Nayar “Polarization-based vision through haze”, Applied Optics, Vol. 42, No. 3. January 2003.    [9] R. Fattal, “Single image dehazing”, International Conference on Computer Graphics and Interactive Techniques, No. 72, pp. 1-9, 2008.    [10] Kenji Mori, Terutoshi Kato, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase, Takayuki Miyahara, Yukimasa Tamatsu, “Visibility Estimation in Foggy Conditions by In-vehicle Camera and Radar”, ICICIC2006, pp. 548-551, Beijing China, 2006/8/30-9/1.
To remove fog from images, related research in recent years includes utilizing atmospheric scattering theory to calculate original parameters formed by image fogging, then utilizing the original parameters to perform an anti-derivative restoration to remove fog from the image. [1] and [2] utilize two images captured at the same place at two different times to obtain different depth information in the two images, then filter out fog based on the depth information. However, this technique requires two images of the environment taken at two different times to perform correction on the image that is already affected by fog. The user first needs to obtain background information for the two images, which makes the method impractical for use with processing of unknown environments.
Another method proposed in [3], [4], and [5] utilizes a depth estimate to find a vanishing point, then utilizes the vanishing point to perform filtering. In [3], a vanishing point region is selected, airlight is calculated from the vanishing point region, and depth heuristics are utilized to calculate depth d. A scattering coefficient β is entered manually. Thus, for use in real-time image/video display applications, an appropriate scattering coefficient β must be calculated to degrade the foggy image. In this way, different scattering coefficients β and depths d can be chosen to reduce the influence of fog on the image. However, calculating appropriate values for the scattering coefficient β and the depth d is difficult. In [4], chromaticity is separated into image chromaticity, light chromaticity, and object chromaticity. Image chromaticity refers to chromaticity captured when a camera captures an image. Light chromaticity refers to light coefficients extracted from the object shined on by light. Object chromaticity refers to an object image obtained after extracting the light coefficients. The method of [4] uses these three chromaticities to perform calculation. The most important part of this method is calculating chromaticities. By calculating the chromaticities, a new chromaticity can be derived. The new chromaticity is obtained through an inverse chromaticity-light space technique. When performing inverse chromaticity-light space processing, the derived result is projected onto Hough space, thereby obtaining suitable light and chromaticity coefficients, before removing the influence of fog. However, in regards to processing speed, all image information must first be projected onto the Hough space of chromaticity-light space during processing of chromaticity-light space. Then, angular calculations are used to obtain appropriate cancelling coefficients, which must be calculated for red, green, and blue (RGB). Thus, a large number of calculations are required, making the method impractical for real-time applications, not to mention that the method introduces an image chromaticity bias.
Image chromaticity, light chromaticity, and object chromaticity are also used in [5] for performing fog removal. First, an image region having greatest contrast must be obtained, then fog removal may be performed. The method utilizes a Markov Random Field to obtain airlight. The method requires approximately five to seven minutes to restore a 640×480 pixel image, making the method unsuitable for real-time applications. In light of traffic monitoring systems requiring real-time processing, image processing speed should be improved while still obtaining a good real-time image cleaning effect.
In [6], a black-and-white camera is mounted on an automobile windshield for performing fog removal processing on images of the road ahead. The camera is first calibrated, and distances between the camera and the nearest and farthest points are obtained. Road width is measured according to road markings, and an extinction coefficient β and a distance d of the image are substituted into a contrast restoration equation. However, this method is ineffective when no road markings are available, e.g. when traveling on the water. Further, a scene captured by the automobile camera will change over time. If the furthest point on the road changes (which occurs frequently), the distance obtained in the method must be adjusted. Thus, adaptive distance tracking must be considered. The method is described for a black-and-white camera. If the method were used in a color camera, the RGB color information will change as the distance changes, causing incongruity of the colors of the image, making the method unsuitable for use in color cameras.
In [7], an image shot by an automobile camera is utilized, and a preceding vehicle in front of the automobile is segmented out of the image. A distance measurement machine of a millimeter-wave radar (mm-W radar) is used to measure distance information between the automobile camera and the preceding vehicle. The method uses machine measurement and variance to measure distance d, uses the distance d to obtain an extinction coefficient β, then performs contrast restoration. Although this method can make effective use of the machine measurement, the method requires a preceding object, such as the preceding vehicle, to obtain the distance measurement. If the scene changes, the distance obtained may not lead to an accurate extinction coefficient β. When traveling on the water, the camera is unable to select a reference object analogous to the preceding vehicle due to heavy waves. Thus, this method would be ineffective if applied to a ship with a camera mounted thereon.
In [8], a polarizing filter is utilized for performing fog removal. However, this method requires two different images, and calculation of a worst polarization and a best polarization, or calculation of a polarizer angle of the image, in order to perform fog removal. Thus, if only one image is available, this method is unusable.
The method of [9] first calculates a degree of change of an image, then utilizes atmospheric scattering theory to perform fog removal. However, the method requires 35 seconds to process one image, making the method unusable in real-time monitoring systems.
In [10], a discrete cosine transform (DCT) is performed based on fog thickness classifications. Then, fog heaviness in the image is classified through high/low-frequency information and Manhattan distance. However, this method is only directed at one particular region of the image, and is not directed to classification of the entire image.