Using digital data to represent images, e.g., for computer applications, has become commonplace. Each image usually comprises pixels arranged in horizontal and vertical rows. Each pixel of an image is normally represented by one or more bits of data, e.g., one byte per pixel. The set of data, e.g., rows and columns of pixel data, which represent an image, is frequently referred to as a bitmap. Thus, images represented and stored in a computer's memory as bitmaps are frequently referred to as bitmap images.
In many computer systems, each image is stored as an independent data set or object which can be accessed and processed independently from data representing other images.
Various known systems store and access the data representing a bitmap image in memory in different ways. Some known systems store the data representing an image in regularly spaced memory locations with a fixed amount of spacing, e.g., one byte, being allocated for the data representing each pixel of an image in memory. In most cases, the distance between memory locations is determined by the amount of data used to represent each pixel. This approach results in a regularly spaced image memory region as opposed to an irregularly spaced image memory region which results when pixel data is spaced at irregular or non-uniform distances within memory.
FIG. 1A illustrates an 8.times.8 grid 12 which represents a block of physical image memory 10. Each of the 64 locations within the grid 12 corresponds to a segment, e.g., byte of memory, in which the data representing a single pixel of an image 11 is stored. Thus, each block of the grid 12 represents a segment of memory where data representing a pixel is stored. Shading in the blocks of FIG. 1 and the other Figures of the present application, is used to illustrate the pixel stored in the memory segment. Thus, the 64 pixel image 11 is illustrated by the cumulative effect of the shading of the 64 blocks of the grid 12. These 64 blocks represent 14 black and 50 white pixels.
In accordance with one known approach for storing data representing an image in regularly spaced image memory regions, the data representing the pixels of the image 11 are loaded into consecutive memory locations within a computer's memory space with the data representing the pixels of each row being entered in sequence. Since there is a fixed amount of data, e.g., one byte is used to represent each pixel, adjacent columns of pixel data will be separated from each other in memory by the amount of data used to represent individual pixels, e.g., one byte. In addition, since there will be a preselected known number of pixels per row for any given image, it is possible to determine the number of memory locations that will separate the beginning of each row of pixels in memory. Accordingly, entry of the image data into memory in the above described manner results in a regularly spaced image memory 10 where there is a fixed predictable relationship between the location of pixels within an image, e.g., as specified by image row and column number, and the location within a block of physical image memory 10 where the data representing each of the individual pixels is stored.
When processing an image, it is often necessary to perform a memory access operation to obtain the data representing each pixel of an image. This generally results in a very high number of memory access operations being performed to process each and every image. For example, to process a 640.times.480 image 307,200 memory access operations may be required. For higher resolution images the number of memory access operations required will be even greater.
Typically, image processing algorithms can be optimized, e.g., in terms of the amount of time required to access memory, when operating on regularly spaced memory regions. However, when operating on irregularly spaced and regions, many known image memory access optimizations will not work. This has led to three general approaches for dealing with the problem of storing and accessing images in memory.
The first of the known approaches uses a slow general purpose pixel access technique which can be used to access both regularly and irregularly spaced image memory. While this approach allows for flexibility in terms of how image data is stored, it does so at the expense of computational efficiency in terms of the number and/or complexity of calculations that must be performed to access image memory locations.
A second known approach uses relatively efficient memory access methods over regularly spaced image memory regions and then the use of special case code to handle memory accesses in irregular image memory regions. Special case code may also be used to perform image processing operations at border regions where neighboring pixel data is not readily available. This approach allows for flexibility in terms of image memory access and image manipulation but it does so at the expense of additional development time in terms of the time required to develop routines for handling irregular memory regions and/or for supporting image border processing operations.
The third known approach to storing and accessing data representing images in memory involves the use of relatively efficient memory access methods over regularly spaced memory regions and disallows irregular memory regions. The third known approach also disallows image processing operations in border regions which require neighboring pixel data.
One known method for efficiently accessing regularly spaced image memory regions is illustrated in FIG. 1B. This method maintains a set of row pointers 20. Each set of row pointers 20 includes one row pointer for each row of pixels in an image 24. Each row pointer points to a beginning physical memory location where the data representing the first pixel of the corresponding row of pixels is stored. When performing an image processing operation, the data representing the pixels of any desired row can be readily accessed by using the physical address obtained using the row pointer and adding an increment thereto to arrive at the desired column.
As discussed briefly above, the processing of images is complicated not only by the large number of memory access operations required to process an image using the known techniques, but also by the need for many image processing operations to use neighboring pixel data as part of the processing operation.
In order to support image processing operations which require the use of neighboring pixel data to be performed in border regions of images, many systems require that padding type operations be performed. Often, this involves copying the entire original image into a new, larger space in memory where the regions surrounding the copy of the original image are padded with valid pixel values. This effectively eliminates the border of the original image. Such a padding operation is illustrated in FIG. 2 wherein the image 11 is copied from the original physical image memory 10 into the larger physical image memory 20. As can be seen in FIG. 2, the new physical image 21, created by the copy and pad operation represented by the arrow 22, includes the original image 11 surrounded by white pixels provided by the padding operation.
Unfortunately, such physical copy and pad operations require a relatively large amount of computational resources to access the original image memory locations and copy the image data. In addition, significant amounts of memory are required to store the padded image 21.
Known techniques for image processing also require relatively large amounts of processing resources and physical memory to perform such simple image processing operations as enlarging and reducing an image.
Referring now to FIG. 3, consider the case of enlarging the image 11 using a known image processing technique. The image processing technique represented by the arrow 32 involves generating a new enlarged image 31 from the original image 11 by substituting multiple pixels for each of the original pixels. In FIG. 3 it can be seen that the enlarging process illustrated by the arrow 32 requires a physical image memory that is NxN times the size of original image space, where N represents the degree of enlargement in the horizontal and vertical directions.
Accordingly, enlarging an image using the known image processing technique is both computationally and memory intensive.
While known techniques for reducing an image, e.g., by downsampling in the horizontal and vertical directions by a factor of N as represented by the arrow 42 in FIG. 4, is less memory intensive than an enlargement operation, it still requires a fair number of computations to be performed. Furthermore, additional physical image memory 40 must be provided for storing the reduced image 41 created by the reduction operation if the original image is to be preserved.
It should be noted that with the vast majority of the prior art image processing techniques which produce a new image from an original image it is necessary to maintain the original image in a distinct portion of physical memory if it is to be used again for further image processing operations. Accordingly, when applying the known image processing techniques, physical memory is frequently required to represent both the original image and the new image created as a result of an image processing operation.
Because of the large amount of memory required to represent images in digital form, the amount of memory required to store and process images remains an important consideration for most image processing applications. Furthermore, the number of computations required to perform an image processing operation and the time required to perform those computations is also important for many applications. This is frequently the case in products involving real time image processing applications and/or cost sensitive consumer products such as video games.
Accordingly, there is a need for improved methods and apparatus for storing, accessing, and processing data representing images which use less memory and/or computational resources than the known techniques. In particular, it is desirable that such methods and apparatus provide a sufficient degree of flexibility to allow a wide range of image processing operations to be performed while being relatively efficient in how they access memory. It is also desirable that the improved methods be capable of performing various routine image processing operations without the need for large amounts of physical memory, e.g., such as the memory frequently used to represent an enlarged image or to support a padding operation.