Generating visualizations and interpretations from high-dimensional data is a 1 common problem in many fields. Two key approaches for tackling this problem approaches. We present a new deep architecture for probabilistic clustering, VarP-7 SOM, and its extension to time series data, VarTPSOM. We show that they achieve  way, a much higher degree of confidence in the findings of the exploration is attained (Keim, 2002).
This paper proposes VarPSOM, a method which utilizes variational autoencoders (VAEs) and clustering techniques based on self-organizing maps (SOMs) to learn clustering of image data (MNIST and Fashion MNIST in particular). An LSTM-based extension termed VarTPSOM is also evaluated on medical time series data. For the most part, the experimental results are promising, and the visualizations are particularly nice.
One of my main points of confusion is with the exposition of the method. To start, the objective presented in Eq. 3 is simply a sum of a variational lower bound and the PSOM clustering loss. Does this have a probabilistic interpretation, e.g., is it a lower bound for a particular generative model? If so, this would be useful to discuss prominently in the paper. If not, it is not clear to me what the authors are gaining from the variational framework. The paragraph at the bottom of page 4 that discusses the "advantages of a VAE over an AE" is not convincing to me. The authors claim that "points with a higher variance in the latent space could be identified as potential outliers and therefore treated as less precise and trustworthy". This isn't demonstrated in the experiments, and to the best of my knowledge, this has not been shown in prior work. If I am mistaken, a citation would be appreciated and should be included in the paper. Additionally, the claim that "the regularization term of the VAE prevents the network from scattering the embedded points discontinuously in the latent space" can also be accomplished with AEs with simple regularization, a standard technique for a wide range of AEs.
Similar comments can be made for the VarTPSOM objective in Eq. 6. Prior work in variational inference for time series, e.g., [1, 2] define a probabilistic time series generative model, from which variational inference naturally prescribes a learning objective. In my opinion, this stands in stark contrast to this work, which takes the VarPSOM objective and simply adds a time series loss on top. This is also a viable approach to building models, but why emphasize variational so much if the method is hardly motivated by anything variational?
I believe that the authors need more thorough experimental comparisons if they wish to demonstrate that their method actually benefits from the variational pieces. Most obviously, I do not believe that any of the comparisons represent the proposed method but with the VAE swapped out for some type of AE? It is my understanding that AE+SOM, SOM-VAE, and DESOM do not represent this exact ablation. VarIDEC performing better than IDEC is a data point in support of this hypothesis, however, this is a comparison of a prior method and not the proposed method.
The related work section mentions that SOM-VAE and DESOM are "likely limited by the absence of techniques used in state-of-the-art clustering methods". Is it possible to address this limitation of prior work? If so, how would this approach compare to the proposed method in terms of implementation and performance? I am not necessarily interested in an actual empirical evaluation, but including this in the related work section would likely be interesting for the reader.
The authors claim in the implementation details that "[s]ince the prior in the VAE enforces the latent embeddings to be compact, it also requires more dimensions to learn a meaningful latent space". Is there a citation for this? My understanding is that posterior collapse leads to VAEs not using additional dimensions even when they are provided, which seems to contradict this claim.
Table 2 seems to have very low NMI numbers across the board, am I reading this incorrectly? Are there prior SOTA numbers that can be included?
Finally, it seems that some of the ideas and motivation in the paper are related to learning discrete structures with variational approaches, e.g., [3, 4]. If the authors agree, it may be appropriate to include some discussion in related work.
[1] Johnson et al, "Composing graphical models with neural networks for structured representations and fast inference". NIPS 2016.
[2] Fraccaro et al, "Sequential neural models with stochastic layers". NIPS 2016.
[3] Tomczak and Welling, "VAE with a VampPrior". AISTATS 2018.
[4] Vikram et al, "The LORACs prior for VAEs: Letting the trees speak for the data". AISTATS 2019.
------
To elaborate on my "Experience Assessment" of "I have read many papers in this area": "this area" in my case refers to amortized variational inference and VAEs, not clustering techniques and SOM.