Panoramic images of scenes are used to provide a viewer with an immersive surround experience, as if the viewer was standing in the center of a scene gazing in several directions. Panoramic images can be produced by taking multiple snapshots of a scene, as a camera is rotated horizontally and possibly also vertically. The snapshots are joined together and projected onto an appropriate geometrical surface, such as a cylinder or sphere, to reproduce an omniview effect.
Digital panoramic images are used inter alia to provide virtual reality experiences over the Internet. These images are normally stored on a host computer, or server, and accessed by users over the Internet. Software panorama production tools take digital snapshots as inputs, and numerically stitch them together and project them onto an appropriate curved surface. The digital snapshots may come from a digital camera, or from scanned in pictures. An example of a software system used to produce panoramic images is assignee's PHOTOVISTA.TM. product, which can be accessed at EQU http://www.livepicture.com.
Interactive panorama viewers comprise software that enables a client computer to display a portion of the panorama, so that a user sees a corresponding part of the surrounding scene. The viewers carry out perspective geometry transformations that project the panorama onto a view window. Although only a portion of the scene can displayed within the view window, the panoramic viewer enables the user to interactively navigate the scene. User navigation includes (i) panning in several directions, which translates the viewable portion of the scene, and (ii) zooming in and out, which magnifies an reduces the scene, effectively changing the field of view. An example of a panorama viewer is assignee's LIVE PICTURE.TM. Viewer product, which can be accessed at EQU http://www.livepicture.com.
A problem in working with panoramic images is the difficulty of editing them. Even such a simple operation as linear scaling is difficult to perform, since the panoramic images are warped onto the shape of a cylinder or sphere. For example, if a 512.times.2,048 pixel digital cylindrical panorama is linearly re-sized to 256.times.1,024 pixels, the result of displaying the re-sized panorama with a panorama viewer will not look real. The re-sized panorama will not be properly warped. This is due to the fact that the pixel values in the 512.times.2,048 panorama were computed by non-linearly projecting digitized snapshots onto a round cylindrical surface at an original sampling rate corresponding to 512 pixels in the vertical dimension and 2,048 pixels in the horizontal dimension. Linearly re-sizing the 512.times.2,048 panorama does not correspond to projecting these digitized snapshots at half the original sampling rate. In fact, to carry out the re-sizing it is necessary to re-project the snapshots onto the cylindrical surface at half the original sampling rate, or to carry out an equivalent mathematical transform.
Compositing images over a panorama is an even more difficult task. If an image to be composited is simply laid down "flat" over a panorama, the resulting composite image, when viewed by a panorama viewer, will be out of perspective with the scene. For example, suppose a panoramic scene is taken as a rotating camera sequentially views four walls of a room. On one of the walls is a picture in a frame. A typical compositing operation would be to overlay a new picture within the picture frame. If this new picture is simply laid down "flat" over the frame, then when a user glides through the scene, the picture will be out of perspective, and it will not have the natural look that a picture in a frame is supposed to have. Similarly, if there is a television in the room, one might want to display a video movie segment within the frame of the television. Again, if the video frames are simply laid down "flat" over the television frame, then the video will be out of perspective, and not have the natural look of a movie playing from a television set within the scene.
Assignee's co-pending U.S. patent application Ser. No. 08/647,018 filed May 8, 1996 and entitled RENDERING PERSPECTIVE VIEWS OF A SCENE USING A SCANLINE-COHERENT LOOK-UP TABLE, the disclosure of which is hereby incorporated by reference, discloses a panorama viewing method and system. It describes an efficient way to provide pan and zoom navigational capabilities, so as to enable a user to have a smooth experience as the user interactively views panoramic images. To ensure smooth navigation it is important that the rendering be optimized so as to yield successive frame data at a re-fresh rate of approximately 30 frames/second.
Assignee's co-pending U.S. patent application Ser. No. 08/922,732 filed Sep. 3, 1997 and entitled A METHOD AND SYSTEM FOR COMPOSITING IMAGES, the disclosure of which is hereby incorporated by reference, describes a way to stitch panoramic snapshots together in a way that compensates for possible different lighting conditions in each snapshot. To reproduce an original scene, it is important that there be no visible signs of stitching between successive snapshots.