The formal understanding of deep learning has made great progress based on quantum many-body physics. For example, the entanglement entropy in quantum many-body systems can interpret the inductive bias of neural network and then guide the design of network structure and parameters for certain tasks. However, there are two unsolved problems in the current study of entanglement entropy, which limits its application potential. First, the theoretical analysis of entanglement entropy was only investigated in the representation of a single object (e.g., an image or a sentence), but has not been well studied in the matching of two objects (e.g., question-answering pairs). Second, the entanglement entropy can not be qualitatively calculated since the exponentially increasing dimension of the matching matrix. In this paper, we are trying to address these two problem by investigating the fundamental connections between the entanglement entropy and the attention matrix. We prove that by a mapping (via the trace operator) on the high-dimensional matching matrix, a low-dimensional attention matrix can be derived. Based on such a attention matrix, we can provide a feasible solution to the entanglement entropy that describes the correlation between the two objects in matching tasks. Inspired by the theoretical property of the entanglement entropy, we can design the network architecture adaptively in a typical text matching task, i.e., question-answering (QA) task. Our adaptive strategy sets a new state-of-the-art performance in TREC-QA task, with 2.9% absolute improvement over a recent model.
In this paper, a method for leveraging entanglement entropy for understanding attention matrices is proposed. Specifically, the paper aims at solving two problems: 1) to study the theoretical analysis of entanglement entropy for the matching of two objects (question-answering pairs), and (2) to qualitatively calculate the matching matrix. The introduced approach is based on fundamental connections between the entanglement entropy and the attention matrix. The main goal of the paper is to show that a low-dimensional attention matrix can be derived from a high-dimensional matching matrix. Results are shown for a text matching task on two datasets (TREC-QA, YAHOO-QA). 
My main concerns with this paper are that the approach is not well described and that the proposed contribution appears quite narrow. Overall it is not clear what the actual contribution of this work really is. Several sections of the paper appear convoluted and could be more concise; the text is often written unnecessarily complicated which makes it hard to follow and to comprehend details. While the experiments seem sound the overall improvement of 2.9% compared to SOTA work appear rather shallow. Therefore, in its current state I cannot recommend accepting this paper. 
Detailed comments: 
- The abstract of the paper is very cryptic and the motivation for the proposed approach is not sufficiently described. 
- The paper is not well-written and difficult to follow. Several sentences and sections are left too unclear. E.g. in the introduction sentences like "... but such an indicator only reflects the intricate correlation structures of each single input object (e.g., an image or a text)" or "This is due to the fact that the tensor product occurs in the quantum many-body function for representing the image and text " are confusing as they come without much context. 
- It is not clear what is meant by "matching problem".
- A few sentences in the introduction are exact copies of the abstract, which makes the text appear redundant. It would help to clearly state what the contribution of this work is, instead of repeating sentences. 
- It is not clear what is meant by "relatively-deeper" and "relatively-shallower" layers.
- The text shows several spelling and grammar issues (such as "for the more complex the inputs"). 
- Several sentences don't make sense and are difficult to read (e.g. "Since our work is mainly for the text matching task of a sentence pair, we briefly introduce a recent Quantum Many-body Wave Function inspired Language Modeling").
- The notation and equations in Section 2 are mostly common knowledge and could be moved to the appendix. 
- In Section 3.1. what is meant by "subsystem in deep neural networks"? Later in the text in becomes more clear. So it would be helpful to rearrange the text. 
- "probability amplitude distributions " is not clear. 
- Sentences like "which often correspond to the important information hidden in the matrix" should be accompanied with a reference. 
- Section 3.2 is titled "Network Design Based on Entanglement Entropy", but the section does not actually describe a network architecture, but instead just describes how to obtain the attention matrix and the sample differences. 
- Section 4.1 (first paragraph): it would help to discuss the connection of many-body wave functions to represent questions and answer sentences as two subsystems more clearly and earlier in the text. 
- Section 4.2 (second paragraph) appears quite repetitive. Many sentences have been used in previous sections of the text. 
- The results and discussions shown in Sections 4.4 and 4.5 are interesting and seem sound. However, it is not clear what exactly are the "adaptive settings for kernels" (e.g. in Figure 2) 
- Limitations of the approach are not sufficiently discussed. 
- It is not clear what is meant by "we will investigate the entanglement entropy under high-order conditions" in the conclusion.