Self-explaining models are models that reveal decision making parameters in an interpretable manner so that the model reasoning process can be directly understood by human beings. General Linear Models (GLMs) are self-explaining because the model weights directly show how each feature contributes to the output value. However, deep neural networks (DNNs) are in general not self-explaining due to the non-linearity of the activation functions, complex architectures, obscure feature extraction and transformation process. In this work, we illustrate the fact that existing deep architectures are hard to interpret because each hidden layer carries a mix of low level features and high level features. As a solution, we propose a novel feature leveling architecture that isolates low level features from high level features on a per-layer basis to better utilize the GLM layer in the proposed architecture for interpretation. Experimental results show that our modified models are able to achieve competitive results comparing to main-stream architectures on standard datasets while being more self-explainable. Our implementations and configurations are publicly available for reproductions †.
This paper proposes a feature leveling technique to improve the self-explaining of deep fully connected neural networks. The authors propose to learn a gated function for each feature dimension for whether to directly send the feature to the final linear layer. The gated function is trained with L0 regularization technique proposed in (Louizos et al., 2017) to encourage more low-level features passed to the final layer. Experimental results on MNIST, California Housing, CIFAR10 show that the proposed method can achieve comparable performance with existing algorithms on sparse neural network training.
Quality:
Overall, the paper is well written with some minor formatting errors. The toy example demonstrates the idea of this paper clearly. However, the novelty of this paper, when compared to NIT, is that the L0 regularization is used to pass the feature to last layer. Considering the self-explaining feature, this work can only explain that some of the input features are suited for final layer, while there is no explanation on the other features since they are used to construct higher level features.
Claririty:
Some parts of this paper are not clear:
1.	Why l_k and h_k has to be disjoint? A feature suited for final classification does not suggest that it can’t be used to construct higher-level feature.
2.	In (4), should not B() be an inverse binary activation function: (1-z)?
3.	Is g(.) the Bernoulli distribution?
4.	In section 5.2, why compare the gradients of a specific input example while one can directly look at z_k, the gated function?
5.	I assume the fully connected layers have bias term. If so, (4) suggests that the gated location will also be added with a learned bias, which is different than what the paper proposes.
Novelty:
The novelty of this paper lies in the sparse training objective becomes passing as many lower-level features to final layer as possible instead of zeros out the intermediate weights. However, the key technique, L0 regularization, has been proposed and used as stated in the related work. While the authors state the application L0 to a novel context to select features is different from prior work, the novelty is rather incremental.  
Significance:
This work demonstrates that the L0 regularization technique for sparse neural network training can also be applied to learn a skip-layer connection. However, from both novelty, performance, and self-explaining perspectives, this work does not introduce much to the field.
Pros:
1.	The paper is well written.
2.	The toy example showcases the issue that this work tries to tackle.
3.	The experimental results show the comparable performance to existing works.
Cons:
1.	The novelty is not sufficient considering the prior works on sparse neural network training.
2.	There are some clarification issues as mentioned before.
3.	The performance is only comparable to existing works.
4.	The self-explaining contribution is not clear since only a few input features can be explained if they are passed to the final layers.
5.	There is no experiment on how \lambda would affect the resulting network architecture.
Minor corrections:
1.	First paragraph on sec. 5: three datasets: two datasets (or mention it’s in appendix).
2.	5.2 compare to NIT: the citations are in wrong format. Also the reference for NIT is corrupted.