Deep neural networks have demonstrated promising prediction and classification performance on many healthcare applications. However, the interpretability of those models are often lacking. On the other hand, classical interpretable models such as rule lists or decision trees do not lead to the same level of accuracy as deep neural networks and can often be too complex to interpret (due to the potentially large depth of rule lists). In this work, we present PEARL, Prototype lEArning via Rule Lists, which iteratively uses rule lists to guide a neural network to learn representative data prototypes. The resulting prototype neural network provides accurate prediction, and the prediction can be easily explained by prototype and its guiding rule lists. Thanks to the prediction power of neural networks, the rule lists from prototypes are more concise and hence provide better interpretability. On two real-world electronic healthcare records (EHR) datasets, PEARL consistently outperforms all baselines across both datasets, especially achieving performance improvement over conventional rule learning by up to 28% and over prototype learning by up to 3%. Experimental results also show the resulting interpretation of PEARL is simpler than the standard rule learning.
Review Summary
--------------
The paper presents a combination of rule lists, prototypes, and deep representation learning to fit classifiers that are said to be simultaneously "accurate" and "interpretable". While the topic is interesting and the direction seems novel, I don't think the work is quite polished or competitive enough to be accepted without significant revision. The major issues include non-competitive evaluation of what "interpretability" means, ROC AUC numbers that are indistinguishable from standard deep learning (RCNN) pipelines that use many fewer parameters, and many unjustified choices inside the method itself. The paper itself could also benefit from revision to improve flow and introduce technical ideas to be more accessible to readers.
Paper Summary
-------------
The paper presents a new method called "PEARL" (Prototype Learning via Rule Lists), which produces a rule list, a set of prototypes, and a deep feed-forward neural network that can embed any input data into a low-dimensional feature space. The primary intended application is classifying subjects into a finite set of possible disorders given longitudinal electronic health records with categorical features observed at T irregular time intervals. 
The paper suggests learning a representation for each subject's data by feeding the EHR time series into a recurrent convolutional NN. The input data is a 2 x T array, with one row representing observed data and second row giving time delay between successive observations. The vector output of an initial convolutional RNN is then fed into a highway network to produce a final vector denoted "h". 
Given an encoder to produce feature vectors, and a fixed rule list learned from data itself, the paper suggests obtaining a prototype for each rule by computing the average vector of all data that matches the given rule. The quality of these prototypes and related neural networks (for computing features and predicting labels from features) is then assessed via their loss function in Eq. 1: a weighted combination of how well the prototypes match the learned embeddings (distance to closest prototype) and how well the classifier predicts labels.  The core idea is that the embedding is learned to classify well while creating a latent space that looks like the prototypes of the rule list.
After training an embedding and NN classifier on a fixed rule list, it seems the data is reweighted according to some heuristic procedure to obtain better properties, then a new rule list is trained and the process repeats again. (I admit the reweight procedure's purpose was never clear to me).
Experiments are done on a proprietary heart failure EHR dataset and on a subset of MIMIC data. 
Strengths
---------
* Seems original: I'm unaware of any other method connecting rule lists AND prototypes AND NNs
* Neat applications to healthcare
Limitations
-----------
* Interpretability evaluation seems weak: no human subject experiments, no quantiative metrics, unclear if rule-lists shown is an apples-to-apples comparison
* Prototypes themselves never evaluated 
* Many design choices inside method not justified with experiments -- why highway networks + RCNNs?
Major Issues with Method
------------------------
## M1: Not clear that AUC difference between PEARL and baselines is significant
The major issue is that the presented approach does not seem significantly different in predictive performance than the baseline Recurrent CNN. Comparing ROC AUC, we have PEARL's 0.688 to RCNN'S 0.682 with stddev of 0.009 on the proprietary heart failure dataset, and PEARL's 0.769 to RCNN's 0.766 with stddev of 0.009. When AUCs match this closely, I struggle to believe one model is definitively better, especially given that the RCNN has 2x *fewer* parameters (8.4k to 18.4k). 
If the counterargument is that the resulting "deep model" is not "interpretable", one should at least compare to a post-processing step where the decision boundary of the RCNN is the reference to which a rule list or decision tree is trained.
## M2: Interpretability evaluation not clear.
Isn't the maximum number of rules set in advance? 
Additionally, prototypes are a key part of this work, but the learned prototypes are not evaluated at all in any figure (except to track avg. distance from prototype while training). If prototypes are so central to this work, I would like to see a formal evaluation of whether the learned prototypes are indeed better (in terms of distance, or inspection of values by an expert, or something else) than alternatives like Li et al.
## M3: Missing a good synthetic/small dataset experiment
Neither of the presented data tasks is particularly easy to understand for non-experts. I'd suggest creating an additional experiment where the audience of ML readers is likely to easily grasp whether a set of rule lists is "good" for the problem at hand... maybe create your own synthetic task or a UCI dataset or something, or even use the stop-and-frisk crime dataset from the Angelino et al. 2018 paper. Then you can compare against just a few relevant baselines (rule lists only or prototypes only). I think a better illustrative experiment will help readers grasp differences between methods. 
## M4: How crucial is feature selection?
In each iteration, Algo. 1 performs feature selection before learning rules. Are any other baselines (trees, rule lists) allowed feature selection before the classifier is learned? What would happen to PEARL without feature selection? What method is used for selection? (A search of the document only has 'feature selection' occur once, in the Alg. itself, so it seems explanation is missing).
## M5: Why are multiple algorithm iterations needed?
Won't steps 3 and 4 of Alg. 1 result in the same rules every time? It's not clear then why on subsequent iterations the algorithm would improve. Perhaps it's just the reweighting of data that causes these steps to change?
Minor issues
------------
## Loss function notation confusing
Doesn't the rule list classifier s_R take the data itself X? Not the learned embedding h(X)? Please fix or clarify Eq. 1. I think you might clarify notation by just writing yhat(h(X)) if you mean the predicted label of some example as done by your NNs. Using "R" makes folks think the rule list is involved.
## Not clear why per-example reweighting is required
None of the experiments assess why per-example reweighting (lines 6-9 of Algo. 1) is required. Readers would like to see a comparison of performance with and without this step.
## Not clear or justified when "averaged" prototypes are acceptable
Are your "averaged" prototypes guaranteed to satisfy the rule they represent? Is taking the average of vectors that match a rule always guaranteed to also match the rule? I don't think this is necessarily true. Consider a rule that says "if x[0] == 0 or x[1] == 0, then ___".  Suppose the only matching vectors are x_A = [0 1] and x_B = [1 0]. The average vector is [0.5 0.5] which doesn't work.
## Several different measures of distance used without careful justification 
Why use two different distances -- Euclidean distance to assess distance to prototypes for prototype assignment, and then cosine similarity when deciding which examples to upweight or downweight? Why not just use Euclidean distance for both (appropriately transformed to a similarity)?
Comments on Presentation
------------------------
Overall I think every section of the paper needs significant revision to improve a reader's ability to understand main ideas. Notation could be introduced slowly (explain purpose and dimension of every variable), assumptions could be clearly stated (e.g. each individual rule can have ANDs but not ORs), and design choices justified. You might try the test of giving the paper to a colleague and having them explain back the ideas of each section to you... currently I do not believe this version passes this test.
The introduction claims that "clinicians are often unwilling to accept algorithm recommendations without clarity as to the underlying reasoning", but I would be careful in blindly asserting this without evidence. For a nice argument about avoiding blind assumptions about what doctor's will and won't accept, see Lipton's 2017 paper "The Doctor Just Won't Accept That" (https://arxiv.org/abs/1711.08037)
Additionally, the authors should clarify more precisely what definition of interpretability is needed for their applications. Is it simplicity? Is it conceptual alignment with known medical facts? Is it the ability to transparently list the rules in plain English?
Line-by-line details
--------------------
## Sec. 2
When introducing p_j, should clarify this this is one prototype vector of many.
When defining p_j = f_j(X), can you clarify what dimensionality p_j has? Is it always the same size as each example's data vector x_i?
## Sec. 3
Fig. 2: I don't find this figure very easy-to-understand. It's clear that after embedding raw features to a new space, the learned rules are *different*, but it's not clear they are *better*.  None of the illustrated rules perfectly segments the different colors, for example. I guess the point is all the red dots are within one rule? But they aren't alone (there are blue and orange dots too), so it's still not clear this would be a better classifier.
For EHR datasets, are you assuming that events are always categorical? And that outcomes "y" are always discrete (one-of-L) variables? Or could y be real-valued?
Eq. 1: You should make notation clearly indicate which terms depend on \theta. Currently it seems that nothing is a function of \theta.
Eq. 1: Do you also find the prototype set P that minimizes this objective? Or is there another way to obtain P given parameters \theta? This is confusing just from reading the eqn.
What size is the learned representation h(X)? Is it a vector?
Eq. 6: Do you really need a "network" to compute the distance to each of the K prototypes? Can't you just compute these distances directly?
## Sec 4
"Mac OS 1.4" : Do you mean Mac OS version 10.4? Not clear this is relevant.
4.3 Case Study: How do I read these rules? Is this rule applied only if ALL conditions are true? or if any individual one is true ("or")? This is unclear.