In probabilistic classification, a discriminative model based on Gaussian mixture exhibits flexible fitting capability. Nevertheless, it is difficult to determine the number of components. We propose a sparse classifier based on a discriminative Gaussian mixture model (GMM), which is named sparse discriminative Gaussian mixture (SDGM). In the SDGM, a GMM-based discriminative model is trained by sparse Bayesian learning. This learning algorithm improves the generalization capability by obtaining a sparse solution and automatically determines the number of components by removing redundant components. The SDGM can be embedded into neural networks (NNs) such as convolutional NNs and can be trained in an end-to-end manner. Experimental results indicated that the proposed method prevented overfitting by obtaining sparsity. Furthermore, we demonstrated that the proposed method outperformed a fully connected layer with the softmax function in certain cases when it was used as the last layer of a deep NN.
This paper proposes a classifier, called SDGM, based on discriminative Gaussian mixture and its sparse parameter estimation. 
The method aims to have a flexible decision boundary due to the mixture model as well as to generalize well owing to the sparse learning strategy. 
While the method incorporates distinct ideas in the literature such as increased model complexity and use of a sparse prior, the paper needs more clearer explanation of the method design and careful empirical investigation. 
My major concerns are as follows. 
1) Explanation of the learning algorithm of Section 2.2 is unsatisfactory. 
Specifically, the objective function and the principle of algorithm are unclear. 
My guess is:
* c, m and T, z are equivalent. (T, z are one-hot representation of c, m.) 
* The objective is to maximize the conditional probability of equation (1) given labeled data (x, t). 
* This maximization is carried out in a similar way to the EM algorithm where m (or z) is regarded as a latent variable. This gives equation (9). 
* A MAP estimate of parameter w is calculated using the prior (8). 
Assuming the points above, I am still unsure about the following points: 
1-a) How is the sparsity induced? 
The prior of w seems to have the l2 regularizer, but how are \pi and r pulled toward zero?
1-b) How is the update of \alpha (17) derived?
Does this strengthen the sparsity by increasing \alpha given small \hat{w}?
What is the "orthogonal component" of \Lambda?
1-c) Is structure in w ignored?
Equation (7) indicates that parameter w has a certain structure such as nonnegative s_{cmii} or the determinant |\Sigma_{cm}| interacting with s_{cmij}. 
In other words, the degree of freedom in w assuming the Gaussian likelihood of x is smaller than H. 
These structure would be violated if the gradient descent or Newton's method is applied. 
Do you mean by *discriminative* that we can freely set the parameter w?
Then, this point should be emphasized. 
2) Which parameter is learned when combined into NN?
Parameters w and \pi? 
How are these parameters made sparse in the end-to-end learning?
3) Ablation test to investigate which aspect impacts the performance. 
Section 4 describes SDGM incorporates disciminative model, mixture model, and sparse Bayesian parameter estimation. 
It is more informative to provide empirical results to see the impact of each property by comparing SDGM with, for example, RVM, discriminative GMM and sparse GMM. 
Some other commets follow.
* \sqrt{\alpha_{cmh}} may be missing from the numerator of (8). 
* For what distribution the expectation with regard to z is taken in e.g. (9, 11, 12, 13, 16)?
* What is D for CIFAR-10 experiment? DenseNet seems to use D=1000 units for the fully connected layer. Did the authors adopt this value?