Published as a conference paper at ICLR 2018 DEEP LEARNING AND QUANTUM ENTANGLEMENT: FUNDAMENTAL CONNECTIONS WITH IMPLICATIONS TO NETWORK DESIGN
Formal understanding of the inductive bias behind deep convolutional networks, i.e. the relation between the network's architectural features and the functions it is able to model, is limited. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning, and use it for obtaining novel theoretical observations regarding the inductive bias of convolutional networks. Specifically, we show a structural equivalence between the function realized by a convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which facilitates the use of quantum entanglement measures as quantifiers of a deep network's expressive ability to model correlations. Furthermore, the construction of a deep ConvAC in terms of a quantum Tensor Network is enabled. This allows us to perform a graph-theoretic analysis of a convolutional network, tying its expressiveness to a min-cut in its underlying graph. We demonstrate a practical outcome in the form of a direct control over the inductive bias via the number of channels (width) of each layer. We empirically validate our findings on standard convolutional networks which involve ReLU activations and max pooling. The description of a deep convolutional network in well-defined graph-theoretic tools and the structural connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.
The paper proposes a structural equivalence between the function realised by a convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which facilitates the use of quantum entanglement measures as quantifiers of a deep network’s expressive ability to model correlations. 
The work is definitely worthwhile digging deeper, bridging some gap and discussions between physics and deep learning. The ultimate goal for this work, if I understands correctly, is a provide a theoretical explanation to the design of deep neural architectures. The paper is well-written (above most submissions, top 10%) with clear clarity. However, removing all the fancy stuff and looking into the picture further, I have several major concerns.
+ Potential good research direction to connect physical sciences (via TN) to deep learning theories (via ConvAC).
- [Novelty is limited and proof is vague] The paper uses physical concepts to establish a "LAYER WIDTHS EFFECT ON THE EXPRESSIVENESS OF A DEEP NETWORK", the core theory (proposed method) part is Section 5 alone and the rest (Section 2,3,4) is for introductory purposes. Putting Theorem 1 in simple, deep learning based English, it says for a dataset with features of a size D, there exists a partition of length scale \epsilon < D, which is guaranteed to separate between different parts of a feature. Based on this, they give a rule of thumb to design the width (i.e., channel numbers) of layers in a deep neural network: (a) layer l = logD is more important than those of deeper layers; (b) among these deeper layers, deeper ones need to be wider, which is derived from the min-cut in the ConvAC TN case. How (a) is derived or implied from theorem 1? 
It seems to me that the paper goes with a rigorous manner till the proof of theorem 1, with all the concepts and denotations well demonstrated. Suddenly when it comes to connecting the practical design of deep networks, the conclusion becomes qualitative without much explanation via figures or visualisation of the learned features to prove the effectiveness of the proposed scheme.
- [Experiments are super weak] The paper has a good motivation and a beautiful story and yet, the experiments are poor to verify them. The reason as to why authors use ConvAC is that it more resembles the tensor operations introduced in the paper. There is a sentence, "Importantly, through the concept of generalized tensor decompositions, a ConvAC can be transformed to a standard convolutional network with ReLU activation and average/max pooling", to tell the relation between ConvAC and traditional convolutions. The theory is based on the analysis of ConvAC, and all of a sudden the experiments are conducted on the traditional convolution. This is not rigorous and not professional for a "technically-sound" paper. How the generalized concepts of tensor decompositions can be applied from ConvAC to vanilla convolutions?
The experiments seem to extend the channel width of *all* layers in a hand-crafted manner (10, 4r, 4r, xxx). Based on the derived rule of thumb, the most important layer in MNIST should be layer 3 or 4 (log 10). Some simple ablative analysis should be:
(i) baseline: fix layer 3, add more layers thereafter in the network;
(ii) fix layer 3, reduce the channel numbers after layer 3.
The (ii) case should be at least comparable to (i) if theorem 1 is correct.
Moreover, to verify conclusion (b) which I mentioned earlier, does the current setting (10, 4r, 4r, xx) consider "deeper ones need to be wider"? What is the value of r? MNIST is a over-used dataset and quite small. I see the performance in Figure 4 (the only experiment result in the paper) just exceeds 90%. A simple trained NN (not CNN) could reach well 96% or so.
More ablative study (convAC or vanilla conv, other datasets, comparison components in width design, etc.) are seriously needed. Otherwise, it is just not convincing to me.
If the authors target on the network design in a more general manner (not just in layer width, but the design of number of filters, layers, etc.), there are already some neat work in this community and you should definitely compare them: e.g., Neural Architecture Search with Reinforcement Learning, ICLR 2017. I know the paper starts with building the connection from physics to deep learning and it is natural to solve the width design issue alone. This is not a major concern.
-------------
We see lots of fancy conceptions, trying to bind interdisciplinary subjects to help analyse deep learning theories over the last few years, especially in ICLR 2017. This year same thing happens. I am not degrading the motivation/intuition of the work; instead, I think it is pretty novel to explain the design of neural nets, by way of quantum physics. But the experiments and the conclusion derived from the analysis make the paper not solid to me and I am quite skeptical about its actual effectiveness.