Recommendation is a prevalent application of machine learning that affects many users; therefore, it is crucial for recommender models to be accurate and interpretable. In this work, we propose a method to both interpret and augment the predictions of black-box recommender systems. In particular, we propose to extract feature interaction interpretations from a source recommender model and explicitly encode these interactions in a target recommender model, where both source and target models are black-boxes. By not assuming the structure of the recommender system, our approach can be used in general settings. In our experiments, we focus on a prominent use of machine learning recommendation: ad-click prediction. We found that our interaction interpretations are both informative and predictive, i.e., significantly outperforming existing recommender models. What's more, the same approach to interpret interactions can provide new insights into domains even beyond recommendation.
This paper is focused on identifying/discovering feature interactions in blackbox models (with a focus on recommendations). Specifically the technique works by first corrupting datapoints and then using this "local dataset" to find interacting features via lasso-regularized multi-level perceptron approach (NID from Tsang et al). Using an expanded feature space and repeated calls to the above steps this is then used to find the most commonly occurring patterns. 
The empirical section is quite interesting, with some nice mix of qualitative and quantitative results. The quantitative results in particular are quite intriguing -- across recommendation and non-recommendation tasks.
Overall the paper makes for an interesting read. On the whole I lean slightly positive -- largely due to the empirical section and the quantitative gains observed by adding the feature interactions as features to the model.
That said I had quite a few concerns about the work:
- In general the exposition was quite lacking in the methodological section. I had to re-read the paper a few times to be able to make out to fully understand the underlying methodology. I would make the overall pipeline very clear and explain out the MADEX pipeline very clearly. The feature expansion section (4.3) in particular was not very clear -- as to how it fit in with the rest of the pipeline and something that could do with more work.  Nomenclature like source and target model are used somewhat arbitrarily and again can be clarified.
- Another concern I had with the approach is how this would work if the underlying features itself were not known. Say for example you had a text understanding model -- the tokenization and vocabulary/OOV etc .. may all be components of the blackbox. Likewise for vision models, what features are being used may vary.
By assuming (effectively perfect) information about the features used, this is no longer really a "blackbox" model. I would have wanted to see some deeper discussion on this topic.
I also had some concerns about the scalability of dense features and their bucketization in this approach. I'm not entirely convinced the method would work as well and efficiently in such feature spaces. I would have wanted to see some more empirical evidence and understanding on this topic.
If possible it would have also been great to understand higher-order interactions and the ability of the model to find them empirically. 
- One thing I wasn't entirely certain of was the amount of novelty in the work since it leverages existing works like NID (Tsang et al) and LIME (Ribeiro et al) for some of the key aspects of the method. It would be nice to see some discussion on this front as well.
- I also would have liked to see significance testing being performed in general across the experiments.
- Another smaller concern stemmed from a lack of mention of the stopping criterion of the model. In particular I'm wondering how models were stopped further training / best checkpoint was picked. This would be good to elaborate on to make sure that the models were all compared fairly from a computational perspective.
- I also had some concerns with the scalability of the approach in large features spaces but am willing to give the authors the benefit of doubt on this one based on the 3 hour number they quoted from their experiments (which indicates something that is not prohibitively slow)