the spatially aware projection pen 560 at step 572 cpu 130 or gpu 115 obtains the data corresponding to the defined path or section cut at step 574 the cpu 130 or gpu 115 renders one or more frames of pixel data to produce two dimensional view s of the three dimensional model at step 576 the rendered frames are stored in image data 155 at step 578 the updated overlay image including the rendered frames is displayed using the projector within the spatially aware projection pen 560 in sum the spatially aware projection pen enables the use of virtual ink in addition to conventional physical ink the virtual ink may be used to capture commands annotate an existing design and communicate with a remote user the virtual ink may be displayed as a projected image on a display surface by the spatially aware projection pen auxiliary design information and rendered images may also be displayed in the projected image the spatially aware feature of the projection pen allows for gestures to be interpreted differently based on the position of the spatially aware projection pen in three dimensional space unlike a conventional digital pen the interaction space available to the user of the spatially aware projection pen is not merely located to the surface input layer but extends to the space above the display surface the integrated projector allows a user to visibly correlate information that is stored inside the pen or on any connected resource with the information illustrated on the display surface as a result paper is no longer just a static source of.