Augmented reality applications permit a user to view a scene that is interlaced with information about the scene. By way of example, as a user manipulates a video camera, an augmented reality application may analyze some of the visual characteristics in the captured scene. If the application is able to obtain more information about an object in the scene based on the visual characteristics, additional information about the object may be displayed on a screen connected to the camera when the video is shown to the user.
A functionality that permits a user to take a picture of a scene and attempts to recognize one or more objects in the scene may be provided. The objects may be quite diverse, e.g., the functionality may compare pictures of buildings to known landmarks, determine the value of bar codes such as a Universal Product Code (UPC), and use optical character recognition (OCR) to extract text from a photo. If an object is recognized, an attempt may be made to obtain additional information about the object where such information exists external to the image data. That additional information may then be displayed to the user or provided to a search engine to identify one or more search results to be shown to the user.