  Fisher Vector classifiers and Deep Neural Networks (DNNs) are popular and
successful algorithms for solving image classification problems. However, both
are generally considered `black box' predictors as the non-linear
transformations involved have so far prevented transparent and interpretable
reasoning. Recently, a principled technique, Layer-wise Relevance Propagation
(LRP), has been developed in order to better comprehend the inherent structured
reasoning of complex nonlinear classification models such as Bag of Feature
models or DNNs. In this paper we (1) extend the LRP framework also for Fisher
Vector classifiers and then use it as analysis tool to (2) quantify the
importance of context for classification, (3) qualitatively compare DNNs
against FV classifiers in terms of important image regions and (4) detect
potential flaws and biases in data. All experiments are performed on the PASCAL
VOC 2007 data set.

Computer Vision and Pattern Recognition