Providing explanations along with predictions is crucial in some text processing tasks. Therefore, we propose a new self-interpretable model that performs output prediction and simultaneously provides an explanation in terms of the presence of particular concepts in the input. To do so, our model's prediction relies solely on a low-dimensional binary representation of the input, where each feature denotes the presence or absence of concepts. The presence of a concept is decided from an excerpt i.e. a small sequence of consecutive words in the text. Relevant concepts for the prediction task at hand are automatically defined by our model, avoiding the need for concept-level annotations. To ease interpretability, we enforce that for each concept, the corresponding excerpts share similar semantics and are differentiable from each others. We experimentally demonstrate the relevance of our approach on text classification and multi-sentiment analysis tasks.
This paper presents a classification model focused on interpretability. The model, Explaining model Decision through Unsupervised Concepts Extraction (EDUCE), is applied to a text classification task, while the authors argue in the appendix that this is also applicable to a wider problem, such as image classification. 
The model is composed of three parts: the first part is detecting salient spans of text relevant to the text classification problem, the second part assigning each salient span a concept label, and the third part which does the classification task based on the binary concept feature label. The models’ loss is composed of two parts: (A) minimizing the cross-entropy of text classification loss and (B) minimizing the cross-entropy of concept classification loss. For (A), as the first and second part of the model introduce discrete choices, they use a RL with Monte-Carlo approximation of gradients. 
The system is evaluated under two measure: 1) classification accuracy and 2) concept accuracy. They define the concept accuracy as follows: after training, they train a classifier that takes output (in the form of <salient span, their concept label>) of the model from the test portion of the data. They split this output into train and test, and report the test accuracy. This aims to show how consistent is the labeling of the salient spans for different methods: if the concept label set correctly merged together semantically similar spans, this “concept accuracy” would be higher. This is a new metric they are proposing. While it is interesting, I would like to see *some* studies on how this correlates with human’s judgements on how interpretable the model is. The paper is introducing a new measure *and* new model, and it’s hard to be persuaded the model is doing well based on this new measure, when there is little ground to know what this measure really measures. 
Overall, I’m not impressed with the models’ performances. The aspect rationale annotated beer sentiment dataset, presented by Lei et al (2016), has provided one of few opportunities to evaluate interpretability / rationale model quantitatively. The paper evaluates on this measure, which is included in the appendix, and the results are pretty disappointing compared to the existing models such as Lei et al’s initial baseline or Bastings et al. While the paper argues this method isn’t necessarily designed for this task unlike the other methods, I’m not sure this is necessarily the case. Bastings et al could be applied to other tasks that model is evaluated on, such as DBPedia and AGNews classification. The difference comes on how easy it is to interpret the methods, as these other rationale-based text processing methods would make use of captured words, while EDUCE would make use of detected “concept” clusters. Currently, the only real baselines are the ablations of its own model. 
Table 3 is quite interesting, different “concepts” capture different aspects fairly well. 
Not having a concept loss actually helps the classification accuracy. Would the concepts learned without concept loss qualitatively very different? This goes back to my original point that their new measure of "concept accuracy" is vague. 
Other comments and Q: 
- Figure (3), the visualization is a bit confusing cause it is unclear whether it is each span is a set of spans or a single span. Also, I would recommend making figures colorblind friendly, if possible. 
Q: what kind of classifier was used for the evaluation metric “concept accuracy” classifier? I don’t think it’s mentioned. 
Q: why are you sampling a test set for DBPedia experiments? Is it for efficiency reason?
Q: how sensitive is model’s performance to the hyper parameters, especially the number of concepts?
Q: the current baseline classifier is a simple BiLSTM one, which definitely perform a lot worse than recent pre-trained LMs such as BERT. Would it be easy to use this method on top of richer representation such as pertained LM outputs?
Q: how would this connects to saliency map literature in computer vision? I guess these would be mostly “a posteriori” explanations? Discussion would be helpful.