The gradient of a deep neural network (DNN) w.r.t. the input provides information that can be used to explain the output prediction in terms of the input features and has been widely studied to assist in interpreting DNNs. In a linear model (i.e., g(x) = wx + b), the gradient corresponds solely to the weights w. Such a model can reasonably locally linearly approximate a smooth nonlinear DNN, and hence the weights of this local model are the gradient. The other part, however, of a local linear model, i.e., the bias b, is usually overlooked in attribution methods since it is not part of the gradient. In this paper, we observe that since the bias in a DNN also has a non-negligible contribution to the correctness of predictions, it can also play a significant role in understanding DNN behaviors. In particular, we study how to attribute a DNN's bias to its input features. We propose a backpropagation-type algorithm "bias back-propagation (BBp)" that starts at the output layer and iteratively attributes the bias of each layer to its input nodes as well as combining the resulting bias term of the previous layer. This process stops at the input layer, where summing up the attributions over all the input features exactly recovers b. Together with the backpropagation of the gradient generating w, we can fully recover the locally linear model g(x) = wx + b. Hence, the attribution of the DNN outputs to its inputs is decomposed into two parts, the gradient w and the bias attribution, providing separate and complementary explanations. We study several possible attribution methods applied to the bias of each layer in BBp . In experiments, we show that BBp can generate complementary and highly interpretable explanations of DNNs in addition to gradient-based attributions.
Summary:
The paper's main contribution is to attribute the bias term seen at the output to each of the input dimension appropriately.  The other claim is that together with gradient information , this could enhance existing explainability methods.
  
The paper considers DNNs with piecewise linear activation functions. Then the final DNN output is a piece linear function. So for any given point x, the point lies in one of the linear pieces and there fore, can be written as a linear model. The gradient term can be computed using back propagation methods (although back propagating keeping the weights fixed and keeping the input as the variable). However there are no know existing works that attribute the bias of the linear piece at the final layer to the input dimensions. This paper provides a method to do it such that when you add the vector contribution of all the dimension in the input - it results in the bias vector at the output layer.
The basic idea is to distribute dimension wise bias attribution  at layer \ell to layer \ell-1 by using N_{\ell} x N_{\ell-1} attribution matrix where N_{\ell} is the dimension of layer \ell. This is done using two methods - one using exponential weights and the other using some sort of variance measure from a fixed average bias.
The authors then show using examples from STL-10 and Imagenet datasets, how the gradients and biases attributed to the inputs compare for explanation purposes.
Strengths:
The notion of attributing final layer biases to input layer is novel. Its important given that it carries important information regarding the final classification output.
Weaknesses:
a) This paper lacks quite a bit on comparison with existing work. For example, LRP (layer wise relevance propagation) has been referred to by the authors. However, there is no comparison with LRP heat maps. This website - http://www.explain-ai.org/ - documents state of the LRP methods with code, videos, papers (some of which have been cited by the authors). I think the authors could produce heat map produced by LRP for all these different examples in page 8. For instance, pls look at Image A in Fig .2 in https://arxiv.org/pdf/1708.08296.pdf. There is 'cup' and a 'volcano' in the same picture. The paper compares gradient based heatmaps and LRP based ones. LRP based ones are very crisp (for this image of course). LRPcode is readily available from a well maintained project page - http://www.explain-ai.org/. Will the heat maps of grad/bias look crisper than LRP ? Other papers and more examples of LRP can be found at http://www.explain-ai.org/
LRP takes the final probability weight at the output layer and assigns recursively to other neurons in the penultimate very similar to what the current paper does for bias. However, the attribution mechanism (the weights) are different. A couple of variations are explored in this survey (https://arxiv.org/pdf/1708.08296.pdf).
b)  This point is related to the first- The authors say "Therefore, the interpretation of the DNN’s behavior on the input data should be exclusive to the information embedded in the linear model." - I disagree a little bit here. It is true that behavior of the DNN on the input is exclusive to the linear piece. However, interpretation of the behavior/ explanation of it is another matter. For example, I quote two methods in the literature that have been used for explanation of an input sample - but does not use the linear piece or the gradient information.
     1)  Pls look at - https://arxiv.org/pdf/1703.02647.pdf - (Figure 3 + Section A.8 in the appendix). The paper used streaming submodularity algorithms to actually assemble a part of the image with everything else "blacked out" to determine which sparse parts of the image are responsible for the final output. They have exhaustive comparisons with LIME too. These methods actually rely on behavior of DNN far away from the actual x to explain the behavior at x. "Zeroing out irrelevant" parts is one of the ways of explaining adopted by LIME and these approaches. Ideally the authors should compare with these too.
 
 2) In this paper - https://arxiv.org/abs/1802.07623 - authors provide pertinent negatives - what should "not be" there in the image so that the label does not change - In fact for MNIST data, this produces very interesting additional explanations that is not produced by methods (including LRP and LIME) that rely on things in the image. Again the explanation is not at all related to the linear piece.
In general, linear pieces are so close by , the nearby movement and possibly change of gradients can also provide useful explanation of the behavior.
Overall - at least the authors must discuss the above references + also survey works that highlights "relevant parts of the image" like LIME and streaming sub modularity etc. Further an actual comparison to LRP (code is easily available) is crucial to evaluate the efficacy of the proposed methods given that the authors of LRP have compared with gradient based methods. Comparison with LIME would also be interesting and desirable.