SpaMHMM: Sparse Mixture of Hidden Markov Models for Graph Connected Entities
We propose a framework to model the distribution of sequential data coming from a set of entities connected in a graph with a known topology. The method is based on a mixture of shared hidden Markov models (HMMs), which are trained in order to exploit the knowledge of the graph structure and in such a way that the obtained mixtures tend to be sparse. Experiments in different application domains demonstrate the effectiveness and versatility of the method.
 The authors propose a method to model sequential data from multiple interconnected sources using a mixture of common pool of HMM*s. The composition of the mixture for each data source is both sparse and similar to that of other data sources that are close to it, with the closeness determined by the weights of the edges of an undirected graph. The introduction section is unfocused and sloppy. HMM*s are well understood models in machine learning but the paper falls short in explaining the particular distributed scenario of interest. The narrative jumps from sports, to neuroscience to wireless communications to fMRI, without mentioning the common denominator. The proposed model section lacks also some focus, jumping from distributed sparse representations to multitask learning. The key concept here seems to be the poorly defined concept of *a sparse HMM mixture over a large dictionary of HMM*s*. The formalization of this rather complicated object is not well justified and leaves a lot of guesswork to the reader. Instead of maximizing the likelihood, an alternative objective function (4) is proposed as maximizing the inner products of posterior probability vectors at each node. The authors probably try to say something sensible but the sloppy notation is not very helpful here. The way authors introduced the graph structure into their cost function creates potentially a flexibility. However, the authors could have spent more energy on explaining why sparseness of the mixtures should be a desirable property for the problems they hope to solve with the model. The graph structure could potentially be used without necessitating sparsity, so opting for sparsity needs to be justified. One also suspects that the authors could have written a clearer generative model instead of modifying the maximum likelihood criterion for learning. This would have enabled the readers to appreciate the generative model better. Moreover, the parameter controls how much effect the graphical interrelations is to have on the final learned parameters of the model. The authors however do not present a detailed examination of empirical results of varying , and only suffice to determine it with cross-validation. A more interpretive stance towards lambda would confer increased understanding of how sparsity functions in this model. The cluster analysis at the end of the experiments indeed provide a more tangible demonstration of how sparsity affects the results obtained and why it might be desirable. Overall, the paper is not of sufficient quality to be presented at ICLR.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
SpaMHMM: Sparse Mixture of Hidden Markov Models for Graph Connected Entities
We propose a framework to model the distribution of sequential data coming from a set of entities connected in a graph with a known topology. The method is based on a mixture of shared hidden Markov models (HMMs), which are trained in order to exploit the knowledge of the graph structure and in such a way that the obtained mixtures tend to be sparse. Experiments in different application domains demonstrate the effectiveness and versatility of the method.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The authors propose a method to model sequential data from multiple interconnected sources using a mixture of common pool of HMM*s. The composition of the mixture for each data source is both sparse and similar to that of other data sources that are close to it, with the closeness determined by the weights of the edges of an undirected graph. The introduction section is unfocused and sloppy. HMM*s are well understood models in machine learning but the paper falls short in explaining the particular distributed scenario of interest. The narrative jumps from sports, to neuroscience to wireless communications to fMRI, without mentioning the common denominator. The proposed model section lacks also some focus, jumping from distributed sparse representations to multitask learning. The key concept here seems to be the poorly defined concept of *a sparse HMM mixture over a large dictionary of HMM*s*. The formalization of this rather complicated object is not well justified and leaves a lot of guesswork to the reader. Instead of maximizing the likelihood, an alternative objective function (4) is proposed as maximizing the inner products of posterior probability vectors at each node. The authors probably try to say something sensible but the sloppy notation is not very helpful here. The way authors introduced the graph structure into their cost function creates potentially a flexibility. However, the authors could have spent more energy on explaining why sparseness of the mixtures should be a desirable property for the problems they hope to solve with the model. The graph structure could potentially be used without necessitating sparsity, so opting for sparsity needs to be justified. One also suspects that the authors could have written a clearer generative model instead of modifying the maximum likelihood criterion for learning. This would have enabled the readers to appreciate the generative model better. Moreover, the parameter controls how much effect the graphical interrelations is to have on the final learned parameters of the model. The authors however do not present a detailed examination of empirical results of varying , and only suffice to determine it with cross-validation. A more interpretive stance towards lambda would confer increased understanding of how sparsity functions in this model. The cluster analysis at the end of the experiments indeed provide a more tangible demonstration of how sparsity affects the results obtained and why it might be desirable. Overall, the paper is not of sufficient quality to be presented at ICLR.
assistant
 The authors propose a method to model sequential data from multiple interconnected sources using a mixture of common pool of HMM*s. The composition of the mixture for each data source is both sparse and similar to that of other data sources that are close to it, with the closeness determined by the weights of the edges of an undirected graph. The introduction section is unfocused and sloppy. HMM*s are well understood models in machine learning but the paper falls short in explaining the particular distributed scenario of interest. The narrative jumps from sports, to neuroscience to wireless communications to fMRI, without mentioning the common denominator. The proposed model section lacks also some focus, jumping from distributed sparse representations to multitask learning. The key concept here seems to be the poorly defined concept of *a sparse HMM mixture over a large dictionary of HMM*s*. The formalization of this rather complicated object is not well justified and leaves a lot of guesswork to the reader. Instead of maximizing the likelihood, an alternative objective function (4) is proposed as maximizing the inner products of posterior probability vectors at each node. The authors probably try to say something sensible but the sloppy notation is not very helpful here. The way authors introduced the graph structure into their cost function creates potentially a flexibility. However, the authors could have spent more energy on explaining why sparseness of the mixtures should be a desirable property for the problems they hope to solve with the model. The graph structure could potentially be used without necessitating sparsity, so opting for sparsity needs to be justified. One also suspects that the authors could have written a clearer generative model instead of modifying the maximum likelihood criterion for learning. This would have enabled the readers to appreciate the generative model better. Moreover, the parameter controls how much effect the graphical interrelations is to have on the final learned parameters of the model. The authors however do not present a detailed examination of empirical results of varying , and only suffice to determine it with cross-validation. A more interpretive stance towards lambda would confer increased understanding of how sparsity functions in this model. The cluster analysis at the end of the experiments indeed provide a more tangible demonstration of how sparsity affects the results obtained and why it might be desirable. Overall, the paper is not of sufficient quality to be presented at ICLR.