Model Agnostic Supervised Local Explanations
Model interpretability is an increasingly important component of practical machine learning. Some of the most common forms of interpretability systems are example-based, local, and global explanations. One of the main challenges in interpretability is designing explanation systems that can capture aspects of each of these explanation types, in order to develop a more thorough understanding of the model. We address this challenge in a novel model called MAPLE that uses local linear modeling techniques along with a dual interpretation of random forests (both as a supervised neighborhood approach and as a feature selection method). MAPLE has two fundamental advantages over existing interpretability systems. First, while it is effective as a black-box explanation system, MAPLE itself is a highly accurate predictive model that provides faithful self explanations, and thus sidesteps the typical accuracy-interpretability trade-off. Specifically, we demonstrate, on several UCI datasets, that MAPLE is at least as accurate as random forests and that it produces more faithful local explanations than LIME, a popular interpretability system. Second, MAPLE provides both example-based and local explanations and can detect global patterns, which allows it to diagnose limitations in its local explanations.
Paper summary  This paper focuses on interpretable machine learning (IML) and proposes an approach (denoted SLIM) for achieving interpretability through example-based and local explanations. The approach combines ideas from local linear models and random forests. It builds on existing work on feature selection and supervised neighborhood selection for local linear modeling. The paper uses recent work in IML to define interpretability and shows via experiments on eight UCI datasets how to maintain high accuracy (through the use of random forests and similar methods) while being able to provide local explanations.  Summary of strengths and weaknesses  - Strengths: the paper is well-written and well-organized. It clearly positions the main idea and proposed approach related to existing work and experimentally demonstrates the effectiveness of the proposed approach in comparison with the state-of-the-art.  - Weaknesses: the research method is not very clearly described in the paper or in the abstract. The paper lacks a clear assessment of the validity of the experimental approach, the analysis, and the conclusions.  Quality  - Your definition of interpretable (human simulatable) focuses on to what extent a human can perform and describe the model calculations. This definition does not take into account our ability to make inferences or predictions about something as an indicator of our understanding of or our ability to interpret that something. Yet, regarding your approach, you state that you are “not trying to find causal structure in the data, but in the model’s response” and that “we can freely manipulate the input and observe how the model response changes”. Is your chosen definition of interpretability too narrow for the proposed approach?  Clarity  - Overall, the writing is well-organized, clear, and concise. - The abstract does a good job explaining the proposed idea but lacks description of how the idea was evaluated and what was the outcome.  Minor language issues  p. 95: “from from” -> “from” p. 110: “to to” -> “how to” p. 126: “as way” -> “as a way” p. 182 “can sorted” -> “can be sorted” p. 197: “on directly on” -> “directly on” p. 222: “where want” -> “where we want” p. 245: “as accurate” -> “as accurate as” Tab. 1: “square” -> “squared error” p. 323: “this are features” -> “this is features”  Originality  - the paper builds on recent work in IML and combines two separate lines of existing work; the work by Bloniarz et al. (2016) on supervised neighborhood selection for local linear modeling (denoted SILO) and the work by Kazemitabar et al. (2017) on feature selection (denoted DStump). The framing of the problem, combination of existing work, and empirical evaluation and analysis appear to be original contributions.  Significance  - the proposed method is compared to a suitable state-of-the-art IML approach (LIME) and outperforms it on seven out of eight data sets. - some concrete illustrations on how the proposed method makes explanations, from a user perspective, would likely make the paper more accessible for researchers and practitioners at the intersection between human-computer interaction and IML. You propose a “causal metric” and use it to demonstrate that your approach achieves “good local explanations” but from a user or human perspective it might be difficult to get convinced about the interpretability in this way only. - the experiments conducted demonstrate that the proposed method is indeed effective with respect to both accuracy and interpretability, at least for a significant majority of the studied datasets. - the paper points out two interesting directions for future work, which are likely to seed future research.