1. Field of the Invention
The present invention relates to multifocus digital image restoration and more specifically to a method and apparatus to, restore a deteriorated image in an image processing system using an image integration technology by adjusting an unfocused part of an image so as to cause all parts of the image to be focused, thereby restoring the original image.
2. Description of the Related Art
Generally, an automatic focus control algorithm, which is used in camera modules of various apparatuses (e.g. digital cameras, camcorders, computer visions, etc.) photographing an image in a digital manner, is closely associated with the depth of a lens. According to the depth of a lens, a mechanical Auto Focus (AF) or out-of-focus scheme and a pan focus scheme may exist.
A mechanical AF scheme is used in an optical system having a shallow depth, and can adjust the distance between a lens module and a sensor so as to control a focal distance between an object and an image sensor—e.g., a Charge Coupled Device (CCD). Such a mechanical AF scheme is generally applied to a large digital camera, etc.
A second scheme,—pan focus—is used in an optical system having a deep depth, and allows an image to be focused on an image sensor when a focal distance is fixed and only a distance of more than a predetermined range is maintained. Such a pan focus scheme is applied to a camera module, etc., of a mobile communication terminal requiring a small size.
FIG. 1 is a block diagram illustrating the construction of a digital automatic focus control device applied to a conventional digital photographing apparatus.
Before a description about FIG. 1 is given, the digital automatic focus control device applied to the conventional digital photographing apparatus (e.g. a digital camera, a camcorder, a computer vision, etc.) employs a mechanical AF scheme and a pan focus scheme.
As illustrated in FIG. 1, the conventional digital automatic focus control device employing a mechanical AF scheme or a pan focus scheme includes an image sensor unit 110, a Point Spread Function (PSF) detector 120 and a Constrained Least Squares (CLS) filter unit 130. Hereinafter, the construction of the digital automatic focus control device will be described in more detail. First, the image sensor unit 110 receives an image including multiple objects such as subjects and senses that the received image may be deteriorated. The deteriorated image is transferred to the PSF detector 120. Such an image sensor unit 110 may be constructed as a CCD type, a Complementary Metal Oxide Semiconductor (CMOS) type or a Contact Image Sensor (CIS) type.
The PSF detector 120 receives the deteriorated image from the image sensor unit 110, detects a PSF necessary for image restoration based on the received deteriorated image, and transfers the detected PSF to the CLS filter unit 130.
The PSF detector 120 further includes a database (not shown) storing a plurality of PSFs preset according to distances from a specific focus position, determines a PSF of the best quality according to distances from the database, and may also transfer the determined PSF to the CLS filter unit 130.
The CLS filter unit 130 forms a CLS image restoration filter by using the PSF transferred from the PSF detector 120, and restores the deteriorated image.
FIG. 2 is a flow diagram illustrating a process by which the PSF detector of FIG. 1 detects the PSF.
In FIG. 2, a case where the PSF detector 120 does not include the database storing the PSFs will be described as an example.
More specifically, when the deteriorated image is input from the image sensor unit 110 (S210), the PSF detector 120 divides the deteriorated image into sub-images of a predetermined size by using a canny filter and a line filter provided therein, and detects the edges of the divided sub-images (S220). Then, the PSF detector 120 detects a representative edge having a pixel value greater than a threshold value (S230). If the representative edge is detected, the PSF detector 120 extracts a unit-step response for the detected representative edge, and detects a PSF based on the extracted unit-step response.
According to the conventional digital automatic focus control method as described above, the input image is divided into sub-images of a predetermined size, each edge direction is estimated, unit-step response functions in each edge direction are computed, and an average of the computed unit-step response functions is calculated, so that the PSF is determined (S240). The PSF determined in this way is transferred to the CLS filter unit 130. The CLS filter unit 130 filters and restores the deteriorated image by using the transferred PSF through the CLS image restoration filter.
However, in the digital automatic focus control device of a mechanical AF scheme applied to the conventional camera module, only a specific part of an image is well focused and remaining parts are unfocused, which is different than a pan focus scheme of a fixed focus system in which all parts of an image are focused. In such a case, a clearly restored image can be obtained in a set position, but the substance of an object cannot be understood in remaining positions.
Further, when the PSF is detected in order to restore the deteriorated image, a complicated calculation must be repeated until the PSF satisfies a predetermined level of reliability. Therefore, significant time is required, and thus the deteriorated image cannot be restored in real-time.