Smoothed Analysis of Discrete Tensor Decomposition and Assemblies of Neurons
We analyze linear independence of rank one tensors produced by tensor powers of randomly perturbed vectors. This enables efficient decomposition of sums of high-order tensors. Our analysis builds upon Bhaskara et al. [3] but allows for a wider range of perturbation models, including discrete ones. We give an application to recovering assemblies of neurons. Assemblies are large sets of neurons representing specific memories or concepts. The size of the intersection of two assemblies has been shown in experiments to represent the extent to which these memories co-occur or these concepts are related; the phenomenon is called association of assemblies. This suggests that an animal's memory is a complex web of associations, and poses the problem of recovering this representation from cognitive data. Motivated by this problem, we study the following more general question: Can we reconstruct the Venn diagram of a family of sets, given the sizes of their`-wise intersections? We show that as long as the family of sets is randomly perturbed, it is enough for the number of measurements to be polynomially larger than the number of nonempty regions of the Venn diagram to fully reconstruct the diagram.
# Review for "Smoothed Analysis of Discrete Tensor Decomposition and Assemblies of Neurons"  This paper models and studies the following problem. The brain is known to store "concepts" (the Eiffel tower, Barak Obama) as collections of neurons which fire simultaneously when the concept is "sensed" -- e.g. when one sees a picture of Barak Obama. Furthermore, if there are associations among concepts -- e.g. one has seen a picture of Barak Obama standing in front of the Eiffel tower -- then the "Barak Obama" neurons will tend to fire with the Eiffel tower ones. The high-level question is: is it possible to efficiently learn a "concept map" of someone's brain by measuring strength of associations among small sets of concepts?  The authors model this question as one of learning a set system from (i.e. the Venn diagram of a collection of sets S_1...S_n over some universe) from the sizes of all $\ell$-wise intersections, for some "reasonable" $\ell$. The main result is a polynomial-time algorithm accomplishing this learning problem, so long as  1. set memberships are slightly randomly perturbed (a la smoothed analysis) and 2. the number of measurements $n^\ell$ is larger than the size of the underlying universe.  The algorithm assembles these $n^\ell$ measurements into a certain $\ell$-tensor, then applies known tensor decomposition algorithms to recover the set system. The catch is that these tensor decomposition results are only known to apply to in a smoothed-analysis setting with *Gaussian* perturbations to the underlying tensor components. This is not a natural kind of perturbation in this discrete, set-system setting. The authors show that these tensor decomposition algorithms also work under a much wider variety of perturbation models, essentially any random perturbation for which the each coordinate's perturbation remains slightly anticoncentrated even conditioned on the perturbations of all other coordinates. This technical result on allowable perturabtion models for smoothed analysis is of interest in its own right -- smoothed analyses for high-dimensional geometric problems are few and far between because they pose significant technical challenges, and the authors seem to have invented some interesting new tools to carry out their analysis.  I am not an expert in this kind of neural modeling, so I do not know whether similar models and algorithmic questions have been addressed in prior literature. I am an expert in tensor decomposition algorithms; in my view the new smoothed analysis represents a NIPS-worth contribution in itself, especially since the application is reasonably interesting. Tensor decomposition is a major tool in provable algorithms for machine learning: expanding the class of smoothed analyses under which tensor decomposition methods are known to work removes unnecessary and unnatural technical assumptions on the kinds of models which can be learned by tensor decomposition.  The writing is clear in the first few pages; I did not have the chance to verify the technical details line-by-line but the approach described seems reasonable. I recommend in favor of accepting this paper to NIPS.