Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We explore an alternative to these approaches by directly regularizing a black-box model for interpretability at training time. Our approach explicitly connects three key aspects of interpretable machine learning: (i) the model's inherent interpretability, (ii) the explanation system used at test time, and (iii) the metrics that measure explanation quality. Our regularization results in substantial improvement in terms of the explanation fidelity and stability metrics across a range of datasets and black-box explanation systems while slightly improving accuracy. Finally, we justify theoretically that the benefits of our regularization generalize to unseen points.
The paper considers the problem of training black box models for improved interpretability, and  proposes to penalize black-box models at training time using two regularizers that correspond to fidelity and stability explanation metrics. As computing the regularizers exactly is computationally intensive they propose two approximating algorithm. In addition, as the one for fidelity is still prohibitive, a randomized variant is proposed. Connections are established between the regularizers and the model's Lipschitz constant or total variation. A generalization bound is presented for local linear explanations. The proposed approach is evaluated on a variety of datasets. 
The paper deals with an important problem and the exposition is clear.  While regularizing deep learning models is a pertinent  direction, I feel the paper makes a couple of overstatements, and overall I am not fully convinced by the approach and empirical evaluation, as outlined below.
- The paper states "recent approaches that claim to overcome this apparent trade-off between prediction accuracy and explanation quality are in fact by-design proposals that impose certain constants on the underlying model families they consider" and that they are addressing this shortcoming. But in fact, the proposed regularizations do exactly the same: they impose certain constraints. Indeed the regularizers encourages models with lower Lipschitz constant or with small total variation across neighborhoods.
- I also find that it is unsurprising that regularizing via fidelity or stability will lead to models with better fidelity/stability so it's an artificial way to yield "improved interpretability" and this is more of an issue because fidelity and stability are kind of proxy metrics to evaluate interpretability.
- It would be important to investigate further the difference between regularization and explanation neighborhoods. This might not be a bad thing which in fact help with generalization. 
- Proposition 1 supports algorithm 2, but it is not a given at all that Algorithm 1 will have smaller local variances across neightborhoods and hence might generalize well.  It would be important to proceed with an empirical study of the local variance across neighborhoods for Algorithm 1.
 
- Computational complexity remains an issue as ExpO-1D-fidelity is performing much worse.
- Comparison against alternative approaches beyond SENN are lacking (e.g. Lee et al 2019, Wang and Rudin,2015 etc).
 
 Overall I feel that more work is needed to convincingly demonstrate the importante of the proposed approach.