1. Field of Invention
The present invention relates to a method for processing and showing digital images. More specifically, the invention is a method for restoring distortive images to normal ones by means of finding out the viewpoint and focal length of a lens in a camera.
2. Related Art
The imaging principle of cameras described by general optical theory is mostly built on hypotheses of a rectilinear projection model. A camera with a similar kind of lens usually can get image pictures close to normal reality. But there is a drawback—the field of view (FOV) is small. For instance, the FOV of standard lenses is around 45 degrees to 55 degrees. It enables some devices that need a broad FOV, like surveillance or endoscopes etc., to use wide-angle lens or fisheye lens (FEL) in order to capture pictures with a big FOV with every single shot.
The FEL is used to capture extremely wide-angled images. Usually, a fisheye lens camera (FELC) can take 180 degrees or more without moving or rotating the camera itself. Coming with the increase of the angle of view, there is also a severe problem of distortion in the captured images. Thus, it is necessary to come out with a calibration method in order to obtain images that are close to reality. The level of accuracy of the calibration method attained determines what fields the FELC can be applied. For instance, if the surveillance systems only need to see where people or things are, they can tolerate a partial distortion in images. If the purpose is taking pictures for virtual reality (VR), it is also acceptable that images “look like” normal ones. However, if our purpose involves 3-D image metering, such as stereoscopic or autonomous robotic vision, it is difficult to obtain accurate 3-D positions of images from reality in situations that some cubical optical parameters of the FEL are unknown.
However, for customers, the kind of lens that has the advantages of wide-angled views and showing image in clarity as well as the capability of metering accurately will be very competitive and attractive. Moreover, with the excellent characteristic of a nearly infinite view depth, it is a point that other kinds of lenses are scarcely comparable to the fisheye lens. If the imaged distortion isn't counted, the fisheye lens is superior to other kinds of lenses. Thus, for expanding the applied fields, how to calibrate the distortive images of the FEL is vitally important.
Currently, there are many calibration methods have been advanced. R. Y. Tasi [1987] (Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses”, IEEE Journal of Robotics and Automation, Vol. RA-3, No 4, August, 1987, pp 323–344) employs five non-coplanar points of known absolute coordinates in viewing space and transfers them by a rotation matrix and a translation matrix to gain the viewpoint and the focal length of a normal lens. The result from Tasi's model is pretty accurate. But its demonstration also is based on the hypothesis of rectilinear projection; it will involve a large error under a severely nonlinear projection system like the FEL. For the purposes of calibrating the FEL, Tasi's model seems not to be a common choice. Thereafter, there is a simple calibration method aimed at the FEL. With reference to FIG. 1A and FIG. 1B, wherein FIG. 1B is a cubical projection of FIG. 1A, it is assumed that the image plane 1 is an ellipse (or a circle), and the intersection point 13 of its major axis 11 and minor axis 12 is exactly the image center (the point C on both the figures). In addition, it is also assumed that the FOV taken by the FEL is exactly 180 degrees. On the premised basis, it is deduced that the periphery of the imaged area has a zenithal angle of π/2, the one on the distortion center 13 has an angle of 0, and the others are determined by the relative location between the point C 13 and the periphery. For example, the point A in FIG. 1A corresponds to the point A′ in FIG. 1B. The method of calibration described above is simple and needs no calibration target to assist. Nevertheless, its premises are not really correct. First, the center point 13 on the obtained picture perhaps is not the real image center. Next, the periphery of an image taken by the FEL is usually blurred, so it is difficult to correctly decide where the imaged boundary exactly lies. Accordingly, the fidelity of the calibrated image is not verifiable. Obviously, this image-based model of analysis cannot be used appropriately in the domain of 3-D metering.
With regard to the patents, TeleBobotic International Inc. has disclosed several technologies regarding the FEL, such as the U.S. Pat. Nos. 5,185,667, 5,313,306, 5,359,363 and 5,384,588. Overall, the contents of the technologies are described as following: a hemispheric FOV is shot by a FELC without moving or rotating the camera, utilizing a dedicated electronic circuit controlled by a computer to digitally transfer the original distortive images into normal perspective ones and show them in a display. The technologies are indicated to apply in the fields of orientation, surveillance, endoscopes, remote control, etc. Subsequently, according to TeleBobotic's disclosure, Interactive Pictures Corporation further brought up a serial of new inventions, such as the U.S. Pat. Nos. 5,764,276, 5,877,801, 5,903,319, 5,990,941, 6,002,430, 6,147,709 and 6,201,574 B1. But no matter whichever above all employs a same projection mode to calibrate images. They do not really find out the viewpoint and the focal length the parameters of the FEL, and consider that the same projection mechanism shown in FIG. 1A and FIG. 1B, the equidistant projection (EDP) model. Within the indistinct situations of the exact optical projective mechanism, the present technologies are only for extending and restoring original images but not improve on accuracy. Applications in the field of 3-D image metering would still be restricted.
In the projective models, besides the EDP there are other FEL projection modes known to those skilled in the art—stereographic projection and orthographic projection (often called an equisolid angle projection). Their respective formulas are presented separately as following:
1. Equidistant projection: IH=fθ
2. Stereographic projection: IH=2f×tan(θ/2)
3. Orthographic projection: IH=f×sin θwhere,                IH: the distance from an imaged point to the optic axis of a lens (also called the image height)        f: the effective focal length of the FEL        θ: the incident angle in the focal plane, being named the zenithal angle if the optical axis is upward as shown in FIG. 1B.        
Theoretically, the stereographic projection is the best, but the equidistant form is a normally given type that is easier to produce. Therefore, most of the current calibration methods have a postulation that almost all projection modes of the FEL are equidistant projection. Basically, it is not certain.
On the other hand, although lenses are usually designed in a specific projective mechanism, after being made it is difficult to verify whether they match the desired specification or not. Furthermore, when the FEL is installed into a real system (such as a camera), its optical parameters such as the effective focal length and the imaged visual angle probably vary accordingly. For this reason, if a simple and common technology is developed, which can verify the optical characteristics of the fabricated devices being produced, to provide a guarantee of quality for the product during sale, it would greatly raise the value.
It is a standpoint to those skilled in the art of the rectilinear projection; the FEL is treated as no “real viewpoint”. However, if the corresponding projection mode can be classified and the focal length can be discovered, it can not only calibrate distortion but also be applied in the field of 3-D metering and in the quality control or the specification verification of the deployed products.