1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method, and can be applied to, for example, electronic still cameras.
2. Description of the Related Art
Hitherto, in image processing apparatuses, such as electronic still cameras, image quality has been improved by various types of image processing. Here, examples of such image processing include a noise reduction process (hereinafter will be referred to as an “NR process”) for reducing a noise level, a skin-tone process (hereinafter will be referred to as an “SK process”) for smoothing a skin-tone portion by using a low-pass filter and improving the appearance of a face of a person, and the like, and a wide-dynamic-range correction process (hereinafter will be referred to as a “WDR process”) for expanding the apparent dynamic range by dynamically varying the gradation in accordance with the luminance distribution of an image.
FIG. 19 is a block diagram showing an electronic still camera of the related art. In an electronic still camera 1, a solid-state image-capturing element 2 is a two-dimensional image sensor formed of, for example, a charge-coupled device (CCD) sensor, a complementary metal-oxide semiconductor (CMOS) sensor, or the like, and performs a photoelectric conversion process on an optical image formed on the image-capturing plane by lenses (not shown) and outputs an image-captured signal S1.
An analog-to-digital (A/D) conversion circuit 3 performs an analog-to-digital conversion process on the image-captured signal S1 and outputs image-captured data D2.
After an optical correction unit 4 corrects image distortion of the image-captured data D2, which is caused by optical characteristics of the lenses, the optical correction unit 4 converts the image-captured data D2 into image data D3 formed of color data of red, green, and blue, and outputs it.
A white-balance unit 5 adjusts the white balance of the image data D3 output from the optical correction unit 4 and outputs it. A wide-dynamic-range correction unit 6 performs a WDR process on image data D4 output from the white-balance unit 5 and outputs it. A skin-tone processor 7 performs an SK process on image data D5 output from the wide-dynamic-range correction unit 6 and outputs it. A noise suppression unit 8 performs an NR process on image data D6 output from the skin-tone processor 7 and outputs it. A gamma processor 9 performs a gamma correction process on image data D7 output from the noise suppression unit 8 and outputs it.
A recorder 10 performs a computation process on image data D8 output from the gamma processor 9, and converts the image data D8 into image data formed of a luminance signal and color-difference signals. Here, when the luminance signal and the color-difference signals are denoted as Y, and Cr and Cb, respectively, and when color data of red, green, and blue is denoted as R, G, and B, respectively, the computation process is represented by, for example, Y=0.3 R+0.6 G+0.1 B, Cr=R−Y, and Cb=B−Y. The recorder 10 records the image data of the luminance signal and the color-difference signals on a recording medium 11. Examples of the recording medium 11 include an optical disc, a magnetic disc, a memory card, and the like.
FIG. 20 is a block diagram showing in detail the wide-dynamic-range correction unit 6, the skin-tone processor 7, and the noise suppression unit 8. In the wide-dynamic-range correction unit 6, a correction value generator 15 generates a gradation correction curve used to correct the gradation of the image data D4 on the basis of the image data D4 output from the white-balance unit 5. More specifically, the correction value generator 15 calculates the distribution of the luminance values of the image data D4, and generates a gradation correction curve so that the luminance level difference between the adjacent gradation values in the luminance level range in which the distribution is concentrated is increased and conversely, the luminance level difference between adjacent gradation values in the luminance level range in which the distribution is not concentrated is decreased.
FIG. 21 is a characteristic curve diagram showing an example of a gradation correction curve. The example of FIG. 21 is an example for when the image data D4 constitutes a high contrast image formed by a backlight or the like. The wide-dynamic-range correction unit 6 sets a γ curve in accordance with the image data D4 and expands the apparent dynamic range so that a low luminance portion is enhanced and a high luminance portion is suppressed. The correction value generator 15 generates a gradation correction curve so that a correction value C1 gradually rises from the value 1 in the image data D4 as a result of an increase in the luminance value, thereafter decreases to less than the value 1, and finally returns to the value 1. The correction value generator 15 generates the gradation correction value C1 on the basis of the luminance values of the image data D4 by using the generated gradation correction curve.
A multiplier 16 multiplies the gradation correction value C1 by the corresponding pixel values of the image data D4, thereby corrects the gradation of the image data D4, and outputs the image data D5.
In the skin-tone processor 7, a skin-tone area detector 17 detects pixels of the skin tone from the image data D5 output from the wide-dynamic-range correction unit 6, and outputs a skin-tone determination flag F1. As shown in FIG. 22, when the hue of the skin-tone portion is viewed in the HSV space, the skin tone varies over a range of 0 to 40 degrees. By using the deviation variation of the hue, the skin-tone area detector 17 generates image data of the color-difference signals Cr and Cb from the image data D5. Furthermore, by determining the signal level of the color-difference signals Cr and Cb, the hue of the image data is determined, the image data D4 belonging to the detection area of the skin tone, which is shown in FIG. 23, is detected, and the skin-tone determination flag F1 is output.
A parameter generator 18 counts, for each pixel of interest, the number of skin-tone determination flags F1 with regard to pixels in a predetermined range in the horizontal direction and in the vertical direction in which the pixel of interest is at the center, thereby obtains the number of skin-tone pixels in the predetermined range and detects the number of skin-tone pixels as the amount of skin tone. As shown in FIG. 24, the parameter generator 18 generates a parameter B whose value increases as the amount of skin tone increases and outputs it. The parameter B is a parameter for controlling the characteristics of a low-pass filter for performing a smoothing process on the skin-tone portion.
A low-pass filter (LPF) 19 decreases a cut-off frequency or increases the amount of attenuation in response to an increase in the parameter B and processes the image data D5, so that setting is performed in such a manner that the amount of suppression of higher frequency components is gradually changed in the contour portion of a face or the like. As a result, the higher frequency components of the image data D5 are suppressed in only the portion of the skin tone. More specifically, the low-pass filter 19 is formed of low-pass filters of two systems, which have different cut-off frequencies and attenuation ratios, and a weighted-addition circuit for performing weighted addition of the low-pass filter outputs of the two systems. The low-pass filter 19 smoothes the image data D5 by using the low-pass filters of the two systems and thereafter performs the weighted addition of the low-pass filter outputs of the two systems by using a weighting coefficient in accordance with the parameter B. Thus, an increase in the parameter B causes the cut-off frequency to be decreased or causes the amount of attenuation to be increased.
On the basis of a noise model of the image data D6, the noise suppression unit 8 performs a smoothing process on the image data D6 and outputs it, and a parameter generator 20 outputs a parameter for controlling the characteristics of the smoothing process. In the example of FIG. 20, the smoothing process is performed by an ε filter 21, and the parameter generator 20 outputs a threshold value ε1 of the ε filter 21 as a parameter for controlling the characteristics of the smoothing process.
The noise model is such that the amount of noise N that is mixed into the image data D6 until the image data D6 is input to the noise suppression unit 8 is formed as a mathematical expression. In the case of the electronic still camera 1, since the analog-to-digital conversion circuit 3 performs an analog-to-digital conversion process on the image-captured signal S1 output from the solid-state image-capturing element 2, a noise model is represented by the following equation:N=a×x1/2+b  (1)where a is the coefficient by light shot noise, b is the level of floor noise, and x is the pixel value of the pixel of interest. In equation (1), the term of a×x1/2 on the right side is the amount of light shot noise that occurs in the photoelectric conversion process in the solid-state image-capturing element 2. b is the amount of noise that is mixed in from the time when stored electric charge of each pixel, which results from a photoelectric conversion process in the solid-state image-capturing element 2, is output as an image-captured signal S1 and an analog-to-digital conversion process is performed thereon by the analog-to-digital conversion circuit 3. Therefore, this noise model is changed by the solid-state image-capturing element 2, the analog-to-digital conversion circuit 3, and the like, which are used.
The parameter generator 20 applies the pixel value of each pixel of interest to the pixel value x of equation (1) in order to determine the amount of noise N for each pixel of interest by using a noise model, and outputs the amount of noise N as a threshold value ε1 of the ε filter 21. FIG. 25 is a characteristic curve diagram showing the relationship between the pixel value of the pixel of interest and the threshold value ε1 by using this noise model.
The ε filter 21 selects image data D6 to be processed on the basis of the threshold value ε1, performs a smoothing process thereon, thereby preventing deterioration of edge components, which are pattern components, and performs a noise suppression process.
In the ε filter 21, as shown in FIG. 26, a pixel-selection flag generator 22 receives, for each pixel of interest, image data of pixels d-4 to d-1, and d1 to d6 in a predetermined range in the horizontal direction and in the vertical direction, in which a pixel d0 of interest is at the center, via a memory (not shown), and detects pixels included in the range of the threshold value ε in which the pixel value of the pixel d0 of interest is at the center. The pixel-selection flag generator 22 outputs a determination flag F2 indicating that the detected pixel is to be processed. Therefore, in the example of FIG. 26, the determination flag F2 is output to the pixel d0 of interest in each of the pixels d-4, d-3, d-1, d1 to d3, d5, and d6 other than the pixels d-2 and d4.
On the basis of the determination flag F2, a low-pass filter (LPF) 23 selects image data to be processed, and performs a smoothing process thereon with respect to the pixel of interest, thereby suppressing the noise level of the image data D6, and outputs it.
At this point, regarding processing of the wide-dynamic-range correction unit 6, Japanese Unexamined Patent Application Publication No. 2001-298621 and corresponding U.S. Pat. No. 6,724,943 disclose a configuration in which edge components are stored and a WDR process is performed. Regarding an SK process, Japanese Unexamined Patent Application Publication No. 2006-41946 discloses a configuration in which occurrence of jitter is prevented and an SK process is performed. Regarding an NR process, Japanese Unexamined Patent Application Publication No. 2005-311455 discloses a configuration in which secondary differentiated values of image data in place of the pixel values of image data are determined using a threshold value ε, and image data to be processed is selected. Regarding an NR process, Japanese Unexamined Patent Application Publication No. 2006-60744 discloses a configuration in which a threshold value of an ε filter is varied using an average value of the pixel values of the pixel of interest and surrounding pixels.
In the processing of image data of the structure shown in FIG. 20, after the gradation of the image data D4 is corrected to be of a non-linear type using a gradation correction curve in the wide-dynamic-range correction unit 6, in the noise suppression unit 8, the noise level is suppressed using the threshold value ε1 in accordance with the noise model. Therefore, in the noise suppression unit 8, there is a problem in that it is difficult to correctly suppress noise by using a noise model due to an influence of the gradation correction curve. As a result, in the configuration shown in FIG. 20, for example, when the low luminance portion is enhanced in the wide-dynamic-range correction unit 6, it becomes difficult to sufficiently suppress the noise level increased in the low luminance portion due to the enhancement of the low luminance portion, and there is a problem in that the noise level increases in the low luminance portion.