Towards Robust Interpretability with Self-Explaining Neural Networks
Most recent work on interpretability of complex machine learning models has focused on estimating a posteriori explanations for previously trained models around specific predictions. Self-explaining models where interpretability plays a key role already during learning have received much less attention. We propose three desiderata for explanations in general -explicitness, faithfulness, and stability -and show that existing methods do not satisfy them. In response, we design self-explaining models in stages, progressively generalizing linear classifiers to complex yet architecturally explicit models. Faithfulness and stability are enforced via regularization specifically tailored to such models. Experimental results across various benchmark datasets show that our framework offers a promising direction for reconciling model complexity and interpretability.
Summary: The paper proposes an alternative approach to obtaining explanations from complex ML algorithms by aiming to produce an explainable modle from the start.  Recently there has been a number of works on interpretability. This work is most similar to concept-based explainability where some of the more recent ones include  • Bau, David, et al. "Network Dissection: Quantifying Interpretability of Deep Visual Representations." 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017. • Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions in AAAI 2018  Conditions on interpretability, as they are mentioned in this paper, have been explored earlier by f.e. Lipton, Zachary C. "The mythos of model interpretability." arXiv preprint arXiv:1606.03490 (2016)., It starts out with a linear regression model and replaces the parameters of the model with a function dependent on the input, adds an optional transformation of the input into a more  low-dimensional space and a generalization of the aggregration into the output. The main novelty of this paper is the idea to start out with an intrinsically interpretable model and extending it. Review:  This  paper is below the acceptance threshold.  They propose training an autoencoder to transform a high-dimensional input into a more interpretable feature space. To construct an appropriate distance metric for the autoencoder (i.e., one that disregards noise and compares the preservation of interpretable features), we would have to have extremely detailed knowledge of the interpretable features for the entire dataset. Otherwise there is no guarantee that the learned features are any more interpretable than the input.  One of their main points is the replacements of the parameter of the linear regression model theta with a function dependent on the input. This function theta(x) is realized with a neural network, a notoriously uninterpretable algorithm. This just moves the uninterpretability but does not provide anymore insight, since the output will still depend on the (uninterpretable) NN.  The authors do propose a difference boundary on theta  for small differences in x.  However, this restricts the interpretable part of the model to a very small input space around the current input. For interpretability we are interested in comparison to the entire data space to gain insight into why this input produces a particular output.   Furthermore, they do not provide precise results or comparisons for their experiments. As an example, for MNIST it is only written that they achieve <3% error rate, which is not competitive for MNIST. For UCI they only mention ‘competitive performance’, when the results could easily have been included in the supplementary.  As such, I would consider the experiments insufficient. Additionally, harder tasks are not included in the experiments.  Some ideas in the paper are promising and I recommend the authors submit this paper to a workshop.  The paper is written  clearly and concisely. The idea of generalizing an interpretable model into a more complex one is interesting and to my knowledge novel.  In the same vein, the proposed conditions for interpretability have not been published in this form before. In my opinion, more experiments with  more complex datasets would greatly benefit this paper as it is unclear to me whether the approach generalizes to more complicated tasks. Minor comments: L. 179: wrong ref approach ?? L. 213 wrong ref