The present invention provides a system and method that permits a user to encounter an augmented reality, such as during shopping, recreation, tourism, or the like. Using a communication device and a data capture device, the user is able to capture data to the data capture device, recognizing defined attributes within the captured data, processing the recognized attributes and returning a response to the person based upon the recognized attributes. The data captured may be single mode or multi-modal data, and may consist, for example of any sensory-type input, e.g., visual, auditory, olfactory, haptics or gustatory obtained by capturing data from an artifact of interest to the user. Artifacts may consist of any type of product, activity, service, or media, or parts thereof that may serve to identify and distinguish the artifact of interest.
SYSTEM AND METHOD FOR AUGMENTED REALITY USING MULTI-MODAL SENSORY RECOGNITION FROM ARTIFACTS OF INTEREST