The present invention relates to computer graphics, and more particularly to interactive graphics systems such as home video game platforms. Still more particularly this invention relates to reconfiguring a 3D graphics pipeline to move hidden surface removal to different locations within the pipeline depending on rendering mode (e.g., alpha thresholding).
Many of us have seen films containing remarkably realistic dinosaurs, aliens, animated toys and other fanciful creatures. Such animations are made possible by computer graphics. Using such techniques, a computer graphics artist can specify how each object should look and how it should change in appearance over time, and a computer then models the objects and displays them on a display such as your television or a computer screen. The computer takes care of performing the many tasks required to make sure that each part of the displayed image is colored and shaped just right based on the position and orientation of each object in a scene, the direction in which light seems to strike each object, the surface texture of each object, and other factors.
Because computer graphics generation is complex, computer-generated three-dimensional graphics just a few years ago were mostly limited to expensive specialized flight simulators, high-end graphics workstations and supercomputers. The public saw some of the images generated by these computer systems in movies and expensive television advertisements, but most of us couldn""t actually interact with the computers doing the graphics generation. All this has changed with the availability of relatively inexpensive 3D graphics platforms such as, for example, the Nintendo 64(copyright) and various 3D graphics cards now available for personal computers. It is now possible to interact with exciting 3D animations and simulations on relatively inexpensive computer graphics systems in your home or office.
A problem graphics system designers are constantly confronting is how to speed up the graphics processing. Reduced image processing time is especially important in real time graphics systems such as interactive home video games and personal computers. Real time systems generally are required to produce thirty new image frames each second.
To achieve higher speed, typical modern 3D graphics systems use a graphics pipeline to render the image. Information specifying an image goes into one end of the pipeline, and the rendered image comes out at the other end of the pipeline. The pipeline includes a number of different processing stages performing the various steps involved in rendering the image (e.g., transformation to different coordinate systems, rasterization, texturing, etc.) all at the same time. Just as you can save time laundering clothes by folding one load of laundry while another load is in the washing machine and still another load is in the dryer, a graphics pipeline saves overall processing time by simultaneously performing different stages of processing as pixels move down the pipeline.
The amount of time it takes for information to get from one end of the pipeline to the other depends on the xe2x80x9clengthxe2x80x9d of the pipelinexe2x80x94that is the number of processing steps the pipeline performs to generate screen pixels for display. Shorter pipelines can process information faster, but image complexity is limited by the reduced number of image processing stages. The additional image processing stages of a longer pipeline can be used to produce more complicated and interesting images at the expense of increased processing time.
A common technique in use in many modern graphics systems today to increase speed performance allows the application programmer (e.g., video game designer) to change the length of the pipeline by turning off graphics pipeline features and processing stages that are not currently being used. For example, the application programmer can selectively turn on and off optional processing operations (e.g., texturing, texture filtering, z buffering, etc.) that take a lot of time to perform. Permitting the application programmer to choose from a menu of processing operations provides great flexibility. If the application programmer is interested in the fastest possible rendering, he or she can select cheaper (in terms of processing time) pipeline processing operations and forego the increased image complexity that would be obtainable by more expensive options. An application programmer interested in more complex images can activate, on an a la carte basis, more complex and expensive functions on an as-needed basis at the cost of increased processing time.
Hidden surface removal is an expensive but important operation performed by nearly all modern 3D graphics pipelines. To create the illusion of realism, it is important for the graphics pipeline to hide surfaces that would be hidden behind other, non-see-through surfaces. Letting the viewer see through solid opaque objects would not create a very realistic image. But in the real world, not every surface behind another surface is hidden from view. For example, you can see objects through transparent objects such as window panes. To provide realism, a 3D graphics pipeline should be able to model transparent objects as well as solid (opaque) objects, and perform hidden surface removal based upon whether or not an object in front of another object is transparent. Modern graphics systems model transparency using an additional channel called the xe2x80x9calpha channelxe2x80x9d and perform xe2x80x9calpha thresholdingxe2x80x9d and alpha blending to achieve transparency and other effects.
One common way to perform hidden surface removal is to use something called a depth buffer. The depth buffer is also called the xe2x80x9cz bufferxe2x80x9d because the z axis is the depth axis. The z buffer typically provides at least one storage location for each pixel (picture element) of the image. When the graphics pipeline writes a pixel on a surface into a color frame buffer that stores the image, it also writes the depth of the surface at that pixel location into a corresponding location in the z buffer. Later, when the graphics pipeline is asked to render another surface at the same image location, it compares the depth of what it has already rendered with the depth of the new surface, relative to the viewpoint. If the new surface is in front of the already rendered surface, the graphics pipeline can discard the new surface informationxe2x80x94since the new surface will be hidden from view. If the depth of the newly presented surface is closer to the viewer, then the graphics pipeline can replace the previously rendered pixel with a new pixel for the new surface because the new surface will hide the previously rendered surface. If the new surface is transparent, then the graphics pipeline may blend the newly presented and previously-rendered surfaces together to achieve a transparency effect.
Since hidden surface removal tends to be a rather expensive operation in terms of speed performance, it is often possible to turn off hidden surface removal at certain times (e.g., while redrawing a background image or drawing certain kinds of geometry that do not require such processing). However, altogether eliminating hidden surface removal is usually not desirable because many or most 3D images require hidden surface removal to provide realism.
The texturing stage is another processing stage found in typical modern graphics pipelines. To provide an increase in image complexity without a corresponding increase in the number of polygons that the graphics pipeline must render, graphics system designers often include the ability to apply two-dimensional textures to polygon surfaces within an image. For example, when creating an image including a tree, it is possible to draw a rectangle or triangle and place a two-dimensional picture or other image of a tree onto that surface. Texturing avoids the need to model each leaf and branch of the tree with one or more polygons, and can therefore substantially save the amount of processing time required to generate images. Another example is rendering a complicated architectural surface such as a brick wall or a wood-grained tabletop. Instead of defining each grain within the tabletop or each brick within the wall, one can image a geometric primitive defining the outlines of the wall or tabletop, and insert an appropriate texture onto that surface to provide a high degree of realism without paying a corresponding speed performance penalty for modeling each brick or each wood grain.
While texture mapping saves processing resources, the texture mapping process is not xe2x80x9cfreexe2x80x9d by any means. Rather, texture mapping can require some relatively time consuming processing (especially when texture filtering is activated), and also requires an additional memory lookup to access the texture information. However, because texturing adds a substantial degree of realism and complexity to an image at relatively low cost, oftentimes the applications programmer wants to take advantage of itxe2x80x94even at the expense of increased processing time.
As can be understood from the above discussion, while various techniques are known for reducing the length of a graphics pipeline, such techniques often require the graphics system designer and/or application programmer to choose between increased image complexity and increased speed performance. Accordingly, further improvements are possible and desirable.
The present invention provides a solution to this problem by providing techniques and arrangements that move the position of z (depth) buffering in a graphics pipeline depending upon rendering mode.
Performing hidden surface removal early in a rasterizing pipeline can shorten the pipeline length for certain image information. Pixels rejected by a depth comparison operation often do not need to be processed by expensive additional processing steps such as texturing because they will be discarded anyway due to the depth comparison. By placing hidden surface removal before additional operations such as texturing, we can discard pixels based on the depth comparison operation and avoid having the pipeline waste valuable time performing expensive operations on those pixels.
On the other hand, some rendering modes require expensive operations such as texturing to be performed before the hidden surface removal operations. For example, if the application programmer has chosen to activate the alpha channel for alpha thresholding (e.g., to model transparency, or provide texture alpha mapping or other alpha-based effects), then hidden surface removal should be delayed until after the alpha operations have been performed. Otherwise, the hidden surface removal operation will not properly take the results of alpha thresholding into account.
In accordance with an aspect of our invention, we provide a reconfigurable graphics pipeline with a hidden surface removal phase that may be placed at different locations within the pipeline depending on pipeline rendering mode. When the pipeline operates in certain rendering modes, the hidden surface removal operation can be performed early in the pipelinexe2x80x94allowing the pipeline to discard obstructed pixels early and avoid wasting its time performing expensive operations on image portions that are obstructed by other portions of the image. For other (e.g., alpha-thresholding-based) rendering modes, the hidden surface removal operation is performed near the end of the pipelinexe2x80x94when the pipeline has developed sufficient additional information to resolve depth comparisons based on such rendering mode.
In accordance with one aspect provided by the invention a graphics pipeline has first and second alternate rendering modes and includes a texturing stage having an input and an output. A reconfiguration arrangement selectively places a hidden surface removal stage alternately at the input or at the output of the texturing stage depending upon the graphics pipeline rendering mode.
In accordance with another aspect of the invention, a method of dynamically reconfiguring a graphics pipeline comprises selectively activating alpha thresholding. If alpha thresholding is not activated, hidden surface removal is performed before texturing. If alpha thresholding is activated, hidden surface removal is performed after texturing.
In accordance with another aspect provided by the invention, a graphics pipeline comprises at least one processing stage including selectively activatable alpha thresholding. This processing stage includes an input and an output. A hidden surface removal stage is selectively coupled to the processing stage input or the processing stage output depending at least in part on whether alpha thresholding is activated.
Another aspect of the invention provides a pixel engine including a first input coupled to a z and edge rasterizer, and a second input coupled to a texture environment unit. A z unit is selectively, alternately coupled to the first input or the second input.
Yet another aspect of the invention provides a method of synchronizing a graphics pipeline reconfiguration comprising:
receiving a command;
stalling a graphics pipeline in response to the received command,
inserting a synchronization token into the graphics pipeline that chases pixels within the graphics pipeline while the pipeline is stalled,
detecting when the synchronization token has reached a predetermined point in the graphics pipeline to confirm that the graphics pipeline has been flushed,
reconfiguring a graphics pipeline in response to such detection to reposition a hidden surface removal stage within the pipeline, and
releasing the stall.
An advantage provided by our invention is that it maintains compatibility with expected image processing protocols and APIs while realizing increased speed performance under certain circumstances. For example, the OpenGL graphics standard requires depth buffering to be performed in response to information developed by an alpha channel. As an example, one way to render translucent objects using OpenGL is to enable depth-buffering (e.g., on a read only basis) while drawing translucent objects. Under that standard, you can draw all of the opaque objects in a scene using depth buffering in normal read/write/update operation, and then preserve those depth values by making the depth buffer read-only. When the translucent objects are drawn, their depth values are still compared to the values established by the opaque objects, so they aren""t drawn if they are behind the opaque ones. However, if they are closer to the viewpoint, they don""t eliminate the opaque objects since the depth-buffer values don""t change. Instead, they are blended with the opaque objects. The present invention accommodates the glDepthiMask type commands implementable under OpenGL still providing the flexibility of moving depth buffering to an earlier stage in the pipeline when translucency or other alpha-based effects are not activated.