Spectral Learning of Large Structured HMMs for Comparative Epigenomics
We develop a latent variable model and an efficient spectral algorithm motivated by the recent emergence of very large data sets of chromatin marks from multiple human cell types. A natural model for chromatin data in one cell type is a Hidden Markov Model (HMM); we model the relationship between multiple cell types by connecting their hidden states by a fixed tree of known structure. The main challenge with learning parameters of such models is that iterative methods such as EM are very slow, while naive spectral methods result in time and space complexity exponential in the number of cell types. We exploit properties of the tree structure of the hidden states to provide spectral algorithms that are more computationally efficient for current biological datasets. We provide sample complexity bounds for our algorithm and evaluate it experimentally on biological data from nine human cell types. Finally, we show that beyond our specific model, some of our algorithmic ideas can be applied to other graphical models.
This paper proposes to solve the parameter learning problem in tree structured HMM using spectral method. Taking advantage of the tree structure, the authors reduced complexity from x^D to x^d where D is the number of nodes in the tree and d is the depth. This represents a significant saving in the case of d << D (for example, for balanced trees we have d = O(log(D)).The paper is well written, where both the goal and the approach is well motivated and the model is clearly explained as well. The authors summarized their contribution in two techniques called the skeletensor and the product projection, both of which are new to me.The experimental results seem strong, although I'm not familiar with the settings and the dataset. One thing that is objectionable is that SMF seems too weak as far as a baseline is concerned. Perhaps regular EM makes a good baseline, since it seems to me that EM can also use the tree structure instead of working in an exponentially large state space by expanding the tree. So I'm not convinced that EM should be dismissed as very slow without providing more arguments. I understand that space is constrained, but experiments on synthetic data, with generic evaluations (like parameter distance, loglikelihod) would help people in the ML community not familiar with the biology task better evaluate this method. Overall, while I'm not entirely sure how important tree HMM for the application, this paper made clear, original progress on an interesting problem, and the techniques developed can probably be used in other settings. This paper should be accepted.