Described is system and method for visual media reasoning. An input image is filtered using a first series of kernels tuned to represent objects of general categories, followed by a second series of sparse coding filter kernels tuned to represent objects of specialized categories, resulting in a set of sparse codes. Object recognition is performed on the set of sparse codes to generate object and semantic labels for the set of sparse codes. Pattern completion is performed on the object and semantic labels to recall relevant meta-data in the input image. Bi-directional feedback is used to fuse the input data with the relevant meta-data. An annotated image with information related to who is in the input image, what is in the input image, when the input image was captured, and where the input image was captured is generated.
Method and apparatus for sparse associative recognition and recall for visual media reasoning