Multi-value Rule Sets for Interpretable Classification with Feature-Efficient Representations
We present the Multi-value Rule Set (MRS) for interpretable classification with feature efficient presentations. Compared to rule sets built from single-value rules, MRS adopts a more generalized form of association rules that allows multiple values in a condition. Rules of this form are more concise than classical singlevalue rules in capturing and describing patterns in data. Our formulation also pursues a higher efficiency of feature utilization, which reduces possible cost in data collection and storage. We propose a Bayesian framework for formulating an MRS model and develop an efficient inference method for learning a maximum a posteriori, incorporating theoretically grounded bounds to iteratively reduce the search space and improve the search efficiency. Experiments on synthetic and realworld data demonstrate that MRS models have significantly smaller complexity and fewer features than baseline models while being competitive in predictive accuracy. Human evaluations show that MRS is easier to understand and use compared to other rule-based models.
Review:  The paper proposes a bayesian rule based model that allows for the inclusion of multiple values of the same feature inside a condition. The model is composed of a set of rules, and an example is classified as positive if it is covered by at least one rule. The generative model has knobs to encourage three interpretability-related properties :  - Small number of rules  - Small number of conditions within each rule  - For each rule, small number of features used (i.e. conditions are clustered into few features)  The improvements of MRS over other rule-based frameworks are compelling. Rule sets that use less features are easier to grasp, and rules with multiple values for the same feature are more intuitive and easier to grasp than the same condition fragmented over multiple conditions.  The paper is mostly clear, but there is a lot of notation. Some suggestions to improve clarity:  - Maybe draw the generative model  - Change the notation of multiple values to [state \in {California, Texas, Arizona, Oregon}] rather than [state =California or Texas or Arizona or Oregon], or in the very least change the spacing around the =  - Drop the second _m in {L_m}_m, {z_m}_m, etc.  - The subscript plus and minus in 4.1 makes the equations very hard to read  Comments / questions:   - Is there a reason for not constraining alpha_+ > beta_+ and alpha_- > beta_- in the formulation? This would avoid the repetition of this condition around Lemma 1.  - It seems that removing a condition reduces the coverage, while adding a condition increases the coverage. These are reversed in 4.2.  - The comparison of number of conditions in Table 1 does not seem entirely fair, as the conditions for MRS are much longer, unless the authors are considering each value in a condition as a separate condition for this count.  - The discussion in lines 326 - 330 does not seem to make sense given that "All of them" is a value included in the condition. This value breaks the intuition presented.  - It's not clear how the authors selected the best model in cross validation after doing grid search. Was the selection done in a systematic or ad hoc way?  Overall, I think this is an interesting interpretable model, with the caveats that it only works for binary classification (and the extensions to more labels is not obvious) and that there are many hyperparameters that need tuning. The results dot not present significant improvements over prior art in terms of accuracy, but there seem to be some improvements in terms of interpretability (although the comparison may not be fair, see comment above).  Typos: 26: primitive concept level -> primitive concepts 51: "an examples" -> examples, "fo example" -> for example 55: for its appealing property (weird phrasing) 68: remove 'the' 73: remove stop before (Millions) 75: 'and it was done…'  weird phrasing 78: various work  79: conjunction connector -> the conjunction connector 81: conjunction/literal -> conjunction / literal (add space) 94: 'simply adapt' -> be adapted 306: uniqeu -> unique 310: 'accuracy 67 conditions' - something is missing here 316: two stops after 'meaningful' 342: MARS -> MRS  Supplementary line 6: MARS -> MRS  # Comments on feedback: - Thanks for addressing the concern about fair comparison - I think that the user study makes the submission stronger, and so do the added results in the new Table 1. I've changed my score to reflect that. - I had understood that 'all of them' is a value like the others. I am question the intuition presented, which states that people prefer to say they are not sure or refuse to answer. This intuition does not fit well with the inclusion of 'all of them' on the right hand side of the rule.