Blossom Tree Graphical Models
We combine the ideas behind trees and Gaussian graphical models to form a new nonparametric family of graphical models. Our approach is to attach nonparanormal "blossoms", with arbitrary graphs, to a collection of nonparametric trees. The tree edges are chosen to connect variables that most violate joint Gaussianity. The non-tree edges are partitioned into disjoint groups, and assigned to tree nodes using a nonparametric partial correlation statistic. A nonparanormal blossom is then "grown" for each group using established methods based on the graphical lasso. The result is a factorization with respect to the union of the tree branches and blossoms, defining a high-dimensional joint density that can be efficiently estimated and evaluated on test points. Theoretical properties and experiments with simulated and real data demonstrate the effectiveness of blossom trees.
The paper continues a line of work in graphical models to go beyondGaussian modeling of real-valued attributes. The apparent novelty isthe combination of a previous algorithm for non-parametric forestdensity estimation, which rests on a non-parametric generalization ofthe classical Chow-Liu algorithm for learning tree graphical models, with the standardgraphical LASSO (glasso), resulting on the authors call a "blossom treegraphical model." The paper states a type of statistical-consistency(theoretical) result, and a few experiments.* On Section 3: Can the authors' comment on how much the stated resultsays about *machine learning* properties (i.e., PAC orlarge-deviation, as opposed to data-size-asymptotic bounds)?  Can theauthors say anything at all about the properties of the constant N?What would be an expression of a lower or upper bound, even if exact?Just like the typical asymptotic convergence as the number of samplesgoes to infinity (e.g., CLT), it is hard to know when thelarge-deviation bounds of the kind provided in the paper start beingvalid. Without such information regarding N and the exact numberof samples n, while it may be an interesting and useful statisticalresult, it does not appear very useful to machine learning, IMHO.Hence, I would have Section 3 moved fully to the supplementarymaterial and replace it with more experimental evaluations. For example,I would have moved the sections Supplementary Simulations 1 and 2 intothe main body of the paper. But even then, a slightly more thoroughevaluation would be useful such as experimenting by changing the classor form of the underlying ground-truth density, and the class ofunderlying graphs, whenever applicable, the number of samples, and thenumber of attributes/variables. *  Given the nature of the paper, a reference to the Chow-Liu algorithm, from 1968, seems warranted, even if only in passing. C. K. Chow, and C. N. Liu, "Approximating Discrete ProbabilityDistributions with Dependence Trees," IEEE Transactions of InformationTheory, vol. IT-14, no. 3, May 1968Despite the different statistical setups/models (i.e., discretevs. continuous), and the fact that the work in the submitted papergoes beyond simple trees/forests/blossom representations, it isundeniable that the core of the underlying approach has its roots onChow-Liu's work.* It'd have been nice to see experiments on real-world data such asthose on microarray data that Liu et al (JMLR, 2011) performed. Isthe problem public access to the data? What about other datasets(e.g., fMRI)?* I am curious as the core distinctions among the following work andforest density estimation, which serves as the foundation for the submission?- Marwan Mattar, and Erik Learned-Miller.  Improved generative models forcontinuous image features through tree-structured non-parametricdistributions. UMass Amherst Technical Report 06-57, 10 pages, 2006.- Mihai Datcu, Farid Melgani, Andrea Piardi, and SebastianoB. Serpico, "Multisource Data Classification With Dependence Trees,"IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 3,pp. 609-617, March 2002. I am interested in directly ML-relevant, as opposed to statistical,distinctions.* It has been quite a long time since Chow and Liu's work in 1968. Hence,I was originally skeptical that no one had actually tried anything like what theauthors' propose. If it exists, I have not been able to find it. So,while somewhat surprising, I must give the authors their due for thenovelty and originality of their proposed approach... Now, I maychange my mind later, should I find a reference :) ... All kidding aside,even if I find an old reference, it does not take much away from theauthors' originality/novelty, because no one in recent years haveproposed the technique, despite the intense attention to the problemof modeling continuous/real-valued attributes going back a decade, at least!