Field of the Invention
The present invention relates to an image processing technology configured to perform image processing for an image generated by image capturing.
Description of the Related Art
An image obtained by an image capturing apparatus, such as a digital camera, may deteriorate due to aberrations, such as a spherical aberration, a coma, a curvature of field, and an astigmatism of an image capturing optical system. These aberrations can be expressed by a point spread function (“PSF”). An optical transfer function (“OTF”) obtained by Fourier-transforming the PSF contains frequency component information of the aberration, and can be expressed by a complex number. A modulation transfer function (“MTF”) is an absolute value or an amplitude component of the OTF, and a phase transfer function (“PTF”) is a phase component of the OTF. The MTF and PTF are frequency characteristics of the amplitude and phase components in the image deterioration caused by the aberration.
Since the OTF of the image capturing optical system affects (deteriorates) the MTF and PTF of the image generated by image capturing, the image becomes a deteriorated image in which each point asymmetrically blurs like the coma. Since the PSF is different according to a color component, such as red, blur, and green, in the image, a blur differs according to the color component and the deteriorated image is a color blurred image.
A known method for correcting a blur (deterioration) in an image uses information of the OTF of the image capturing optical system. This method is also referred to as an image recovery or an image restoration, and a process for correcting (reducing) the deteriorated image using the OTF information of the image capturing optical system will be referred to as an “image restoration process” hereinafter. One conventional method of the image restoration process is a method of convoluting an image restoration filter in a real space having an inverse characteristic of the OTF with an input image, although it will be described in detail later.
A more accurate OTF of the image capturing optical system is necessary for a more effective image restoration process. The OTF can be obtained through a calculation using design value information on an image capturing optical system. The OTF can be also calculated by Fourier-transforming an intensity distribution of a captured point light source.
The OTF changes according to an image capturing condition, such as an F-number and a focal length (zooming state) of an image capturing optical system, and an image height on an image capturing plane. The OTF also changes according to a focus state of the image capturing optical system for the object. Therefore, the image restoration filter used for the image restoration process is different according to a position (image height) in the image and a distance to the object contained in the image (or the focus state in the image capturing optical system). When the same image restoration filter is applied to the entire image, a false color occurs in a partial area having a different characteristic of the image restoration due to the OTF that differs caused by the image height and the focus state. Thus, in order to perform a good image restoration process for the entire deteriorated image, it is necessary to apply a proper image restoration filter to each partial area in the designated image.
A data amount of the OTF to be stored is enormous if the proper image restoration filter is generated for each partial area of the designated image. Japanese Patent Laid-Open No. (“JP”) 2012-073691 discloses a method for reducing a data amount to be stored by storing coefficient data used to reconstruct an OTF for each image pickup condition and for each image height. However, this method causes a heavy calculation burden so as to reconstruct the OTF for each image height and for each focus state using the coefficient data.
U.S. Pat. No. 4,410,804 discloses an image capturing apparatus in which one pixel in an image sensor includes one micro lens and a converter that is divided into a plurality of areas. The divided photoelectric converter receives light fluxes from partial areas (partial pupil areas) that are different from one another in the exit pupil in the image capturing optical system via one micro lens. A plurality of parallax images having parallaxes that are different from one another can be generated through the plurality of partial pupil areas by using the photoelectric conversion signals output from these divided photoelectric converters. Light field (LF) data as a spatial distribution and an angular distribution of the light intensity information can be acquired using the plurality of thus generated parallax images. An image that focuses on a virtual imaging plane different from an image capturing plane in capturing the plurality of parallax images can be generated using the LF data.
JP 2001-083407 discloses an image capturing apparatus configured to generate one captured image by summing up all photoelectric conversion signals from the above divided photoelectric converters.
As described above, in order to generate the proper image restoration filter in the focus state at each of all image heights in the deteriorated image, the OTF data to be stored is enormous and a calculation burden becomes heavy.