Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures
The resemblance between the methods used in studying quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and a request of high bond dimension. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multipartite entanglement renormalization ansatz (MERA). This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning. While keeping the TN unitary in the training phase, TN states can be defined, which optimally encodes each class of the images into a quantum many-body state. We study the quantum features of the TN states, including quantum entanglement and fidelity. We suggest these quantities could be novel properties that characterize the image classes, as well as the machine learning tasks. Our work could be further applied to identifying possible quantum properties of certain artificial intelligence methods.
 Authors of this paper derived an efficient quantum-inspired learning algorithm based on a hierarchical representation that is known as tree tensor network, which is inspired by the multipartite entanglement renormalization ansatz approach where the tensors in the TN are kept to be unitary during training. Some observations are: The limitation of learnability of TTN strongly depends on the physical indexes and the geometrical indexes determine how well the TTNs approximate the limit; TTNs exhibit same increase level of abstractions as CNN or DBN; Fidelity and entanglement entropy can be considered as some measurements of the network. Authors introduced the two-dimensional hierarchical tensor networks for solving image recognition problems, which suits more the 2-D nature of images. In section 2, authors stated that the choice of feature function is arbitrary, and a specific feature map was introduced in Section 4. However, it is not straightforward to connect (10) to (1) or (2). It is better to clarify this connection because some important parameters such as the virtual bond and input bond are related to the complexity of the proposed algorithm as well as the limitation of learnability. For example, the scaling of the complexity O(dN_T(b_v^5 + b_i^4)) is not easy to understand. Is it related to specific feature map? How about the complexity of eigen-decomposition for one tensor at each iterates. And also, whether the tricks used to accelerate the computations will affect the convergence of the algorithm? More details on these problems are required for readers’ better understanding. From Fig 2, it is difficult to see the relationship between learnability and parameters such input bond and virtual bond because it seems there are no clear trends in the Fig 2(a) and (b) to make any conclusion. It is better to clarify these relationships with either clear explanation or better examples. From Fig 3, authors claimed that TN obtained the same levels of abstractions as in deep learning. However, from Fig 3 only, it is hard to make this conclusion. First, there are not too many differences from Fig 3(a) to Fig 3(e). Second, there is no visualization result reported from deep learning on the same data for comparison. Hence, it is not convincing to draw this conclusion only from Fig 3. In Section 4.2, what strategy is used to obtain these parameters in Table 1? In Section 5, it is interesting to see more experiments in terms of fidelity and entanglement entropy.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures
The resemblance between the methods used in studying quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and a request of high bond dimension. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multipartite entanglement renormalization ansatz (MERA). This approach overcomes scalability issues and implies novel mathematical connections among quantum many-body physics, quantum information theory, and machine learning. While keeping the TN unitary in the training phase, TN states can be defined, which optimally encodes each class of the images into a quantum many-body state. We study the quantum features of the TN states, including quantum entanglement and fidelity. We suggest these quantities could be novel properties that characterize the image classes, as well as the machine learning tasks. Our work could be further applied to identifying possible quantum properties of certain artificial intelligence methods.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Authors of this paper derived an efficient quantum-inspired learning algorithm based on a hierarchical representation that is known as tree tensor network, which is inspired by the multipartite entanglement renormalization ansatz approach where the tensors in the TN are kept to be unitary during training. Some observations are: The limitation of learnability of TTN strongly depends on the physical indexes and the geometrical indexes determine how well the TTNs approximate the limit; TTNs exhibit same increase level of abstractions as CNN or DBN; Fidelity and entanglement entropy can be considered as some measurements of the network. Authors introduced the two-dimensional hierarchical tensor networks for solving image recognition problems, which suits more the 2-D nature of images. In section 2, authors stated that the choice of feature function is arbitrary, and a specific feature map was introduced in Section 4. However, it is not straightforward to connect (10) to (1) or (2). It is better to clarify this connection because some important parameters such as the virtual bond and input bond are related to the complexity of the proposed algorithm as well as the limitation of learnability. For example, the scaling of the complexity O(dN_T(b_v^5 + b_i^4)) is not easy to understand. Is it related to specific feature map? How about the complexity of eigen-decomposition for one tensor at each iterates. And also, whether the tricks used to accelerate the computations will affect the convergence of the algorithm? More details on these problems are required for readers’ better understanding. From Fig 2, it is difficult to see the relationship between learnability and parameters such input bond and virtual bond because it seems there are no clear trends in the Fig 2(a) and (b) to make any conclusion. It is better to clarify these relationships with either clear explanation or better examples. From Fig 3, authors claimed that TN obtained the same levels of abstractions as in deep learning. However, from Fig 3 only, it is hard to make this conclusion. First, there are not too many differences from Fig 3(a) to Fig 3(e). Second, there is no visualization result reported from deep learning on the same data for comparison. Hence, it is not convincing to draw this conclusion only from Fig 3. In Section 4.2, what strategy is used to obtain these parameters in Table 1? In Section 5, it is interesting to see more experiments in terms of fidelity and entanglement entropy.
assistant
 Authors of this paper derived an efficient quantum-inspired learning algorithm based on a hierarchical representation that is known as tree tensor network, which is inspired by the multipartite entanglement renormalization ansatz approach where the tensors in the TN are kept to be unitary during training. Some observations are: The limitation of learnability of TTN strongly depends on the physical indexes and the geometrical indexes determine how well the TTNs approximate the limit; TTNs exhibit same increase level of abstractions as CNN or DBN; Fidelity and entanglement entropy can be considered as some measurements of the network. Authors introduced the two-dimensional hierarchical tensor networks for solving image recognition problems, which suits more the 2-D nature of images. In section 2, authors stated that the choice of feature function is arbitrary, and a specific feature map was introduced in Section 4. However, it is not straightforward to connect (10) to (1) or (2). It is better to clarify this connection because some important parameters such as the virtual bond and input bond are related to the complexity of the proposed algorithm as well as the limitation of learnability. For example, the scaling of the complexity O(dN_T(b_v^5 + b_i^4)) is not easy to understand. Is it related to specific feature map? How about the complexity of eigen-decomposition for one tensor at each iterates. And also, whether the tricks used to accelerate the computations will affect the convergence of the algorithm? More details on these problems are required for readers’ better understanding. From Fig 2, it is difficult to see the relationship between learnability and parameters such input bond and virtual bond because it seems there are no clear trends in the Fig 2(a) and (b) to make any conclusion. It is better to clarify these relationships with either clear explanation or better examples. From Fig 3, authors claimed that TN obtained the same levels of abstractions as in deep learning. However, from Fig 3 only, it is hard to make this conclusion. First, there are not too many differences from Fig 3(a) to Fig 3(e). Second, there is no visualization result reported from deep learning on the same data for comparison. Hence, it is not convincing to draw this conclusion only from Fig 3. In Section 4.2, what strategy is used to obtain these parameters in Table 1? In Section 5, it is interesting to see more experiments in terms of fidelity and entanglement entropy.