Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models. We propose Mahé, a novel approach to provide Model-agnostic hierarchicaléxplanations of how powerful machine learning models, such as deep neural networks, capture these interactions as either dependent on or free of the context of data instances. Specifically, Mahé provides context-dependent explanations by a novel local interpretation algorithm that effectively captures any-order interactions, and obtains context-free explanations through generalizing contextdependent interactions to explain global behaviors. Experimental results show that Mahé obtains improved local interaction interpretations over state-of-the-art methods and successfully provides explanations of interactions that are contextfree.
Summary
=======
The authors extended the linear local attribution method LIME for interpreting black box models by non-linear functions to more accurately approximate black box models locally and identifying interactions between model input variables using the previously published neural interaction detection (NID) framework. They further propose a method to discern between context-dependent and context-free interactions. I found the paper hard to understand without being familiar with previously published literature on detecting interactions and could not understand their approach to detect context-free interactions as well as some aspects of their evaluation. I have also concerns about the practically of their method due to the high runtime and the notion of locality in the light of high-dimensional inputs. In the following, I will briefly summarizing my major criticism and give further details below.
Summary of major criticism
=====================
1) The paper is hard to understand without being familiar with previously published literature in the field.
The authors do not describe how they define the interaction sets X_I in equation (3).
2) I could not understand their approach for detecting context-free interactions (section 4.2).
3) It is unclear how discrete variables are locally modified. Can the approach be used for a combination of differently distributed variables, e.g. categorical and continuous variables?
4) Evaluation metrics such as MSE and R-precision are not described and I could not understand other important aspects of their evaluation, e.g. superpixels (figure 6), why and how they modified network architectures for evaluating context-free interactions (section 5.3), or how they asked Amazon Medical Turk users.
5) The authors did not compare the runtime of Mathe with LIME, which is presumably high due to the need of fitting multiple non-linear models (equation 3) and sampling the local neighborhood.
6) The authors  did not compare the total number of model parameters of Mathe with LIME, which might also account for the higher accuracy (lower MSE). 
7) The authors did not evaluate the accuracy and runtime of Mathe on high-dimensional inputs, e.g. large images.
Details
=====  
Abstract
---------------------------
1. The abstract is hard to understand since ‘context-dependent’ and ‘context-free’ are undefined. The authors should also not use ‘dependencies’ as synonym for ‘interactions’.
Introduction
---------------------------
2. The difference between interactions and context-free and context dependent interactions is unclear. Is a variable without interactions context-free, e.g. Buffalo does not interact with water, and a variable with interactions context-dependent? Also, ‘classes of data’ is unclear, which can be misinterpreted as class labels for classification problems. The authors should also clarify what they mean by ‘performance and generality’ in the last section of the introduction.
Section 3.1
---------------
3. The description of the model function f(x) and attribution scores phi(x) is unclear. Since f(x) is undefined when it is first mentioned after equation (1), I suggest to first define f(x) and afterwards define phi(x). The purpose of the attribution score function phi(x) is also unclear without prior knowledge. The authors should more clearly describe that f(x) is the target function (model) of interest, e.g. a classifier, and phi(x) locally approximates f(x) and is interpretable in contrast to f(x).
Section 4.1
---------------
4. The authors should justify why they are sampling x ~ N(x, sigma * I), which assumes that data instances are iid Normal. This is not the case, e.g., if x is categorical or contains a combination of categorical and continuous variables, and variables are correlated. How is sigma chosen? How many samples are used depending on the dimension of x?
5. The authors should briefly describe the basic idea of NID.
6. How are points sampled from the epsilon neighborhood of x? What is a link function?
7. The subsection ‘Hierarchical Interaction Attribution’ is hard to understand without being familiar Tsang et al. The authors should give an example of a hierarchical explanation with different layers. 
Section 4.2
---------------
8. How is the ‘local vicinity’ defined? Which distance metric is used? This is in particular problematic if x is high-dimensional due the the curse of dimensionality. How are continuous and categorical variables locally modified? Did the authors meant to use a lowercase ‘k’ in equation (4), i.e. ‘phi_k =’ instead of ‘phi_K’? I find this section hard to understand without being familiar with the cited literature.
Section 5.1
---------------
9. The number of local vicinity samples is unclear. Did the authors use 1k local vicinity samples for synthetic experiments (Table 1) and 5k samples for real-word synthetic experiments?
10. What is the dimensionality (number of words, characters, or pixels) of real world datasets? This is important since it influences the number samples that are required to approximate the vicinity of a particular data point. It is in particular interesting to know how the model accuracy and runtime depends on the dimensionality and the number of local vicinity samples.
11. The authors should define the evaluation metrics (MSE, R-precision) in addition to citing them.
12. How did the authors choose the interaction sets X_I in equation (3) and (4)? How many MLPs (functions g(.)) did the authors fit to learn phi(x)? Is the number of MLPs the same for LIME and Mathe? Otherwise the performance gain of Mathe over LIME can also be attributed the increased number of models and model parameters (ensemble size). 
13. What is the average training time of Mathe and baseline models on the different datasets?
Section 5.2.2
-----------------
14. How did the authors choose sigma (0.4, 6, 0.4) for the different datasets?
15. How did Amazon medical turk users evaluate Mathe vs. LIME interactions? Were they given for each sentence the best Mathe and best LIME interaction and asked to decide which one is better? Figure 4 should be more clearly described in the caption. The sentence ‘The result of this experiment is that the majority of preferred explanation …’ is unclear and unjustified by only showing one example in Figure 4.
Section 5.2.3
-----------------
16. The authors should discuss figure 6. The results indicate that neither LIME nor Mathe is able to clearly identify the object of interest, e.g. the water buffalo, and interactions, e.g. between the buffalo and water.
Section 5.3
---------------
17. The sentence ‘... the presence of a French word for “this” or “that”, cet, which …’ is unclear. I suggest to give an example to illustrate which interactions are supposed to be detected. The modification of the Transformer model and the reason why this is necessary is unclear. Overall, I find this evaluation unclear and insufficient since it only applies to a particular interaction.