Described is a system for classifying novel objects in imagery. In operation, the system extracts salient patches from a plurality of unannotated images using a multi-layer network. Activations of the multi-layer network are clustered into key attribute, with the key attributes being displayed to a user on a display, thereby prompting the user to annotate the key attributes with class label. An attribute database is then generated based on user prompted annotations of the key attributes. A test image can then be passed through the system, allowing the system to classify at least one object in the test image by identifying an object class in the attribute database. Finally, a device can be caused to operate or maneuver based on the classification of the at least one object in the test image.
Machine vision system for recognizing novel objects