summaries: Quantum neural networks have two parts: a quantum embedding circuit that takes in classical data and embeds it into a quantum state, and a variational quantum circuit that learns a quantum circuit for evolving the quantum state before measurement.  The authors propose to encode the classical input data into the parameterized angle in the quantum embedding circuit using a tensor-train network (instead of a dense neural network). This results in a 3-4% improvement in prediction accuracy for MNIST (87.12% accuracy). This work proposes an end-to-end learning framework TTN-VQC for quantum neural networks. The main problem is that current quantum computer cannot handle large number of qubits.  The idea of this paper is applying QTT to perform dimensional reduction followed by quantum encoding TPE, which can be combined with existing quantum neural network (VQC) as an end-to-end learning model.  As it was already studied in the literature, one way to implement a machine learning classifier in quantum computers is to convert classical input data like images into quantum states that are fed to a Variational Quantum Circuit (VQC). However, current quantum computers have a small number of qbits, which requires to perform a dimensionally reduction of the input datasets as a preprocessing step. In this paper, the authors propose to use data compression based on the Tensor Train Network (TTN) model, which is a very well-known compression technique. They compare TTN compression against a straightforward dense layer and the PCA-based dimension reduction techniques. The paper includes experimental results on MNIST dataset by simulating quantum circuits with up to 8qbits for noisy and noiseless scenarios. Summary:  This paper is dedicated to designing an end-to-end learning framework for quantum neural networks. The authors design a novel quantum tensor network for dimension reduction and quantum embeddings generation, since it is quite related to the practical usage in real-world applications where only a small number of qubits could be supported on available NISQ computers at this moment. The key contribution of this paper is to leverage a tensor train network (TTN) to replace the dense layer for dimension reduction, which enables an end-to-end training process fully conducted in a quantum computer.
abstractText: The advent of noisy intermediate-scale quantum (NISQ) computers raises a crucial challenge to design quantum neural networks for fully quantum learning tasks. To bridge the gap, this work proposes an end-to-end learning framework named QTN-VQC, by introducing a trainable quantum tensor network (QTN) for quantum embedding on a variational quantum circuit (VQC). The architecture of QTN is composed of a parametric tensor-train network for feature extraction and a tensor product encoding for quantum embedding. We highlight the QTN for quantum embedding in terms of two perspectives: (1) we theoretically characterize QTN by analyzing its representation power of input features; (2) QTN enables an end-to-end parametric model pipeline, namely QTN-VQC, from the generation of quantum embedding to the output measurement. Our experiments on the MNIST dataset demonstrate the advantages of QTN for quantum embedding over other quantum embedding approaches.
references.authors: Andrew R Barron
references.title: Approximation and Estimation Bounds for Artificial Neural Networks
references.venue: Machine Learning,
references.authors: Marcello Benedetti Erika Lloyd Stefan Sack Mattia Fiorentini
references.title: Parameterized Quantum Circuits as Machine Learning Models
references.venue: Quantum Science and Technology,
references.authors: Ville Bergholm Josh Izaac Maria Schuld Christian Gogolin M Sohaib Alam Shahnawaz Ahmed Juan Miguel Arrazola Carsten Blank Alain Delgado Soran Jahangiri
references.title: Pennylane: Automatic Differentiation of Hybrid Quantum-Classical Computations
references.venue: arXiv preprint arXiv:1811.04968,
references.authors: Samuel Yen-Chi Chen Chih-Min Huang Chia-Wei Hsing Ying-Jer Kao
references.title: Hybrid QuantumClassical Classifier Based on Tensor Network and Variational Quantum Circuit
references.venue: arXiv preprint arXiv:2011.14651,
references.authors: Samuel Yen-Chi Chen Chao-Han Huck Yang Jun Qi Pin-Yu Chen Xiaoli Ma Hsi-Sheng Goan
references.title: Variational Quantum Circuits for Deep Reinforcement Learning
references.venue: IEEE Access,
references.authors: George Cybenko
references.title: Approximation by Superpositions of A Sigmoidal Function
references.venue: Mathematics of Control, Signals and Systems,
references.authors: Li Deng Jinyu Li Jui-Ting Huang Kaisheng Yao Dong Yu Frank Seide Michael Seltzer Geoff Zweig Xiaodong He Jason Williams
references.title: Recent Advances in Deep Learning for Speech Research at Microsoft
references.venue: In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing,
references.authors: Yuxuan Du Min-Hsiu Hsieh Tongliang Liu Dacheng Tao
references.title: Expressive Power of Parametrized Quantum Circuits
references.venue: Physical Review Research,
references.authors: Yuxuan Du Min-Hsiu Hsieh Tongliang Liu Dacheng Tao Nana Liu
references.title: Quantum Noise Protects Quantum Classifiers Against Adversaries
references.venue: Physical Review Research,
references.authors: Vedran Dunjko
references.title: Inside Quantum Black Boxes
references.venue: Nature Physics,
references.authors: Vedran Dunjko Hans J Briegel
references.title: Machine Learning & Artificial Intelligence in the Quantum Domain: A Review of Recent Progress
references.venue: Reports on Progress in Physics,
references.authors: Vedran Dunjko Jacob M Taylor Hans J Briegel
references.title: Quantum-Enhanced Machine Learning
references.venue: Physical Review Letters,
references.authors: Edward Farhi Jeffrey Goldstone Sam Gutmann
references.title: A Quantum Approximate Optimization Algorithm
references.venue: arXiv preprint arXiv:1411.4028,
references.authors: David H Freedman
references.title: Hunting for New Drugs with AI
references.venue: Nature, 576(7787):S49–S53,
references.authors: Yangguang Fu Mingyue Ding Chengping Zhou
references.title: Phase Angle-Encoded and Quantum-Behaved Particle Swarm Optimization Applied to Three-Dimensional Route Planning for UAV
references.venue: IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans,
references.authors: Xun Gao Zhengyu Zhang Luming Duan
references.title: An Efficient Quantum Algorithm for Generative Machine Learning
references.venue: arXiv preprint arXiv:1711.02038,
references.authors: Timur Garipov Dmitry Podoprikhin Alexander Novikov Dmitry Vetrov
references.title: Ultimate Tensorization: Compressing Convolutional and FC Layers Alike
references.venue: arXiv preprint arXiv:1611.03214,
references.authors: Maxwell Henderson Samriddhi Shakya Shashindra Pradhan Tristan Cook
references.title: Quanvolutional Neural Networks: Powering Image Recognition with Quantum Circuits
references.venue: Quantum Machine Intelligence,
references.authors: Hsin-Yuan Huang Michael Broughton Masoud Mohseni Ryan Babbush Sergio Boixo Hartmut Neven Jarrod R McClean
references.title: Power of Data in Quantum Machine Learning
references.venue: Nature Communications,
references.authors: Thomas Huckle Konrad Waldherr Thomas Schulte-Herbrüggen
references.title: Computations in Quantum Tensor Networks
references.venue: Linear Algebra and its Applications,
references.authors: William Huggins Piyush Patil Bradley Mitchell K Birgitta Whaley E Miles Stoudenmire
references.title: Towards Quantum Machine Learning with Tensor Networks
references.venue: Quantum Science and Technology,
references.authors: Forrest N Iandola Song Han Matthew W Moskewicz Khalid Ashraf William J Dally Kurt Keutzer
references.title: SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and
references.venue: MB Model Size. arXiv preprint arXiv:1602.07360,
references.authors: Sofiene Jerbi Casper Gyurik Simon Marshall Hans J Briegel Vedran Dunjko
references.title: Variational Quantum Policies for Reinforcement Learning
references.venue: arXiv preprint arXiv:2103.05577,
references.authors: Xuanyu Jin Jiajia Tang Xianghao Kong Yong Peng Jianting Cao Qibin Zhao Wanzeng Kong
references.title: CTNN: A Convolutional Tensor-Train Neural Network for Multi-Task Brainprint Recognition
references.venue: Proc. IEEE Transactions on Neural Systems and Rehabilitation Engineering,
references.authors: John Jumper Richard Evans Alexander Pritzel Tim Green Michael Figurnov Olaf Ronneberger Kathryn Tunyasuvunakool Russ Bates Augustin Žı́dek Anna Potapenko
references.title: Highly Accurate Protein Structure Prediction with AlphaFold
references.authors: Abhinav Kandala Antonio Mezzacapo Kristan Temme Maika Takita Markus Brink Jerry M Chow Jay M Gambetta
references.title: Hardware-Efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets
references.authors: Iordanis Kerenidis Jonas Landman Anupam Prakash
references.title: Quantum Algorithms for Deep Convolutional Neural Networks
references.venue: In Proc. International Conference on Learning Representations,
references.authors: Frank Leymann Johanna Barzen
references.title: The Bitter Truth About Gate-Based Quantum Algorithms
references.venue: In the NISQ Era. Quantum Science and Technology,
references.authors: Jin-Guo Liu Lei Wang
references.title: Differentiable Learning of Quantum Circuit Born Machines
references.venue: Physical Review A,
references.authors: Seth Lloyd Maria Schuld Aroosa Ijaz Josh Izaac Nathan Killoran
references.title: Quantum Embeddings for Machine Learning
references.venue: arXiv preprint arXiv:2001.03622,
references.authors: Jarrod R McClean Jonathan Romero Ryan Babbush Alán Aspuru-Guzik
references.title: The Theory of Variational Hybrid Quantum-Classical Algorithms
references.venue: New Journal of Physics,
references.authors: Jarrod R McClean Sergio Boixo Vadim N Smelyanskiy Ryan Babbush Hartmut Neven
references.title: Barren Plateaus in Quantum Neural Network Training Landscapes
references.venue: Nature Communications,
references.authors: Kosuke Mitarai Makoto Negoro Masahiro Kitagawa Keisuke Fujii
references.title: Quantum Circuit Learning
references.venue: Physical Review A,
references.authors: Valentin Murg Frank Verstraete Örs Legeza Reinhard M Noack
references.title: Simulating Strongly Correlated Quantum Systems with Tree Tensor Networks
references.venue: Physical Review B,
references.authors: Michael A Nielsen Isaac Chuang
references.title: Quantum Computation and Quantum Information
references.authors: Alexander Novikov Dmitry Podoprikhin Anton Osokin Dmitry Vetrov
references.title: Tensorizing Neural Networks
references.venue: In Proc. Advances in Neural Information Processing Systems,
references.authors: Román Orús
references.title: Tensor Networks for Complex Quantum Systems
references.venue: Nature Reviews Physics,
references.authors: Ivan V Oseledets
references.title: Tensor-Train Decomposition
references.venue: SIAM Journal on Scientific Computing,
references.authors: Mateusz Ostaszewski Lea M Trenkwalder Wojciech Masarczyk Eleanor Scerri Vedran Dunjko
references.title: Reinforcement Learning for Optimization of Variational Quantum Circuit Architectures
references.venue: arXiv preprint arXiv:2103.16089,
references.authors: Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen Zeming Lin Natalia Gimelshein Luca Antiga
references.title: Pytorch: An Imperative Style, High-Performance Deep Learning Library
references.venue: In Proc. Advances in Neural Information Processing Systems,
references.authors: John Preskill
references.title: Quantum Computing in the NISQ
references.venue: Era and Beyond. Quantum,
references.authors: Jun Qi Hu Hu Yannan Wang Chao-Han Huck Yang Sabato Marco Siniscalchi Chin-Hui Lee
references.title: Exploring deep hybrid tensor-to-vector network architectures for regression based speech enhancement
references.venue: arXiv preprint arXiv:2007.13024,
references.authors: Jun Qi Hu Hu Yannan Wang Chao-Han Huck Yang Sabato Marco Siniscalchi Chin-Hui Lee
references.title: Tensor-to-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network
references.venue: In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing,
references.authors: Valeria Saggio Beate E Asenbeck Arne Hamann Teodor Strömberg Peter Schiansky Vedran Dunjko Nicolai Friis Nicholas C Harris Michael Hochberg Dirk Englund
references.title: Quantum Speed-ups in Reinforcement Learning
references.venue: In Quantum Nanophotonic Materials, Devices, and Systems 2021,
references.authors: Maria Schuld Nathan Killoran
references.title: Quantum Machine Learning in Feature Hilbert Spaces
references.venue: Physical Review Letters,
references.authors: Maria Schuld Francesco Petruccione
references.title: Supervised Learning with
references.venue: Quantum Computers,
references.authors: Maria Schuld Ilya Sinayskiy Francesco Petruccione
references.title: An Introduction to Quantum Machine Learning
references.venue: Contemporary Physics,
references.authors: Pierre Sermanet David Eigen Xiang Zhang Michaël Mathieu Rob Fergus Yann LeCun
references.title: Overfeat: Integrated Recognition, Localization and Detection Using Convolutional Networks
references.venue: In Proc. International Conference on Learning Representations,
references.authors: Andrea Skolik Sofiene Jerbi Vedran Dunjko
references.title: Quantum Agents in The Gym: A Variational Quantum Algorithm for Deep Q-Learning
references.venue: arXiv preprint arXiv:2103.15084,
references.authors: Eric Smalley
references.title: AI-Powered Drug Discovery Captures Pharma Interest
references.venue: Nature Biotechnology,
references.authors: Andrei N Soklakov Rüdiger Schack
references.title: Efficient State Preparation for A Register of Quantum Bits
references.venue: Physical review A,
references.authors: Andros Tjandra Sakriani Sakti Satoshi Nakamura
references.title: Compressing Recurrent Neural Network with Tensor-Train
references.venue: In Proc. International Joint Conference on Neural Networks,
references.authors: Hanrui Wang Yongshan Ding Jiaqi Gu Yujun Lin David Z Pan Frederic T Chong Song Han
references.title: Quantumnas: Noise-Adaptive Search for Robust Quantum Circuits
references.venue: arXiv preprint arXiv:2107.10845,
references.authors: Chao-Han Huck Yang Jun Qi Samuel Yen-Chi Chen Pin-Yu Chen Sabato Marco Siniscalchi Xiaoli Ma Chin-Hui Lee
references.title: Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition
references.venue: In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing,
references.authors: Yinchong Yang Denis Krompass Volker Tresp
references.title: Tensor-Train Recurrent Neural Networks for Video Classification
references.venue: In Proc. International Conference on Machine Learning,
references.authors: R Yu S Zheng A Anandkumar Y Yue
references.title: Long-term Forecasting Using Tensor-Train RNNs
references.venue: arXiv preprint arXiv:1711.00073,
sections.heading: 1 INTRODUCTION
sections.text: The state-of-the-art machine learning (ML), particularly based on deep neural networks (DNN), has enabled a wide spectrum of successful applications ranging from the everyday deployment of speech recognition (Deng et al., 2013) and computer vision (Sermanet et al., 2014) through to the frontier of scientific research in synthetic biology (Jumper et al., 2021). Despite rapid theoretical and empirical progress in DNN based regression and classification (Goodfellow et al., 2016), DNN training algorithms are computationally expensive for many new scientific applications, such as new drug discovery (Smalley, 2017), which requires computational resources that are beyond the computational limits of classical hardwares (Freedman, 2019). Fortunately, the imminent advent of quantum computing devices opens up new possibilities of exploiting quantum machine learning (QML) (Biamonte et al., 2017; Schuld et al., 2015; Schuld & Petruccione, 2018; Schuld & Killoran, 2019; Saggio et al., 2021; Dunjko, 2021) to improve the computational efficiency of ML algorithms in the new scientific domains.
  Although the exploitation of quantum computing devices to carry out QML is still in its initial exploratory stages, the rapid development in quantum hardware has motivated advances in quantum neural networks (QNN) to run in noisy intermediate-scale quantum (NISQ) devices (Preskill, 2018; Huggins et al., 2019; Huang et al., 2021; Kandala et al., 2017). A NISQ device means that not enough qubits could be spared for quantum error correction, and the imperfect qubits have to be directly used at the physical layer. Even though, a compromised QNN approach is proposed by employing hybrid quantum-classical models that rely on the optimization of variational quantum circuits (VQC) (Benedetti et al., 2019; Mitarai et al., 2018). The resilience of the VQC based models to certain types of quantum noise errors and high flexibility concerning coherence time and gate requirements (McClean et al., 2018) admit many practical implementations of QNN on NISQ devices (Chen et al., 2020b; Yang et al., 2021; Du et al., 2020; 2021; Skolik et al., 2021; Dunjko et al., 2016; Jerbi et al., 2021; Ostaszewski et al., 2021). One notable limitation in the current QNN training pipeline is that the quantum embedding is not fully realizable in a quantum computer, which may impede the learning of the QNN. Hence, this work proposes QTN-VQC to enable an end-toend trainable QNN, including data embedding to quantum measurements, that are easily realizable in quantum devices, where QTN stands for the quantum tensor network (Orús, 2019; Huckle et al., 2013; Biamonte et al., 2017; Murg et al., 2010) for generating quantum embedding.
  As shown in Figure 1, our QNN builds a unitary linear operator that consists of three main components: (1) quantum embedding generation; (2) variational quantum circuit; (3) measurement. Quantum embedding generation, also known as quantum encoding, applies a fixed unitary linear operator Hx transforming classical vectors x to quantum states |ψx〉 in a Hilbert space. This step is an important aspect of designing quantum algorithms that directly impact the entire computation cost of VQC and owns a characteristic of quantum superposition. Moreover, the VQC comprises two types of quantum gates: (1) Controlled-NOT (CNOT) gates; (2) learnable parametric quantum gates. The CNOT gates ensure the property of quantum entanglement through mutually connecting the qubits, and the parametric quantum gates can be adjustable to best fit the quantum input states. The model parameters of VQC should be optimized by employing variants of gradient descent algorithms during the training process. Those parametric quantum gates of VQC are similar to the weights assigned to DNN, and such quantum circuits have been justified to be resilient to quantum noises (Farhi et al., 2014; Kandala et al., 2017; McClean et al., 2016). Besides, the measurement M(|gθ(x)〉) aims at projecting the quantum output states |gθ(x)〉 to one classical output zi. This work focuses on quantum embedding generation because it is quite related to the practical usage in machine learning applications in terms of computational cost and representation capability of classical input features. In particular, we design a novel quantum tensor network (QTN) for quantum embedding generation. More specifically, the QTN consists of a tensor-train network (TTN) for dimension reduction and a quantum tensor encoding framework for outputting quantum embeddings. The dimension reduction is a necessary procedure before the quantum encoding because only a small number of qubits could be supported on available NISQ computers at this moment. A typical approach for dimension reduction relies on a classical fully-connected layer, also known as a dense layer, to convert high-dimensional input vectors y into low-dimensional ones x. However, since a dense layer cannot be physically mapped on a quantum computer, much overhead has to be incurred by frequently communicating between classical and quantum devices during the end-to-end training pipeline.
  As shown in Figure 2 (b), one of our contribution is to leverage a tensor train network (TTN) to replace the dense layer in Figure 2 (a). The benefits of applying TTN arise from two aspects: (1) TTN can maintain the representation power of the dense layer, which will be justified in our theorems; (2) TTN is a tensor network and can be flexibly placed in quantum computers, which enables an end-to-end training process fully conducted in a quantum computer. Moreover, in this work, a tensor product encoding (TPE) is delicately designed for generating quantum embedding, which builds the relationship between a classical vector x and the corresponding quantum state |x〉; Besides, we further investigate the representation of QTN-VQC in terms of model size and nonlinear activation function used in TTN. We denote a QTN as the combination of TTN and TPE and utilize QTN-VQC as a genuine end-to-end learning framework for QNN.
sections.heading: 2 RELATED WORK
sections.text: The work (Schuld & Petruccione, 2018; Biamonte et al., 2017; Dunjko & Briegel, 2018) demonstrate that VQC shows great promise in surpassing the performance of classical ML. Prominent examples of VQC based models include quantum approximate optimization algorithm (QAOA) (Farhi et al., 2014), and quantum circuit learning (QCL) (Mitarai et al., 2018). Various architectures and geometries of VQC have been shown in tasks ranging from image classification (Henderson et al., 2020; Chen et al., 2020a; Kerenidis et al., 2020) to reinforcement learning (Chen et al., 2020b).
  As for quantum embedding, basis encoding is the process of associating classical input data in the form of binary strings with the computational basis state of a quantum system (Leymann & Barzen, 2020). Similarly, amplitude encoding is a technique involving encoding data into the amplitudes of a quantum state (Soklakov & Schack, 2006). Unfortunately, the computational cost of both quantum embedding and amplitude encoding becomes exponentially expensive with the increasing number of qubits (Schuld & Killoran, 2019). A new technique of angle embedding makes use of the quantum gates to generate quantum states (Fu et al., 2011), but it cannot deal with the high-dimensional feature inputs. Therefore, this work exploits the use of TTN for dimension reduction followed by a TPE for generating quantum embedding.
  In particular, this work employs the TTN for dimensionality reduction. The TTN model based on TT decomposition in neural networks was first proposed in (Oseledets, 2011), and it could be flexibly extended the convolutional neural network (CNN) (Garipov et al., 2016) and recurrent neural network (RNN) (Tjandra et al., 2017). The empirical study of TTN on machine learning tasks shows that TTN is capable of maintaining the DNN baseline results (Qi et al., 2020a; Yu et al., 2017; Yang et al., 2017; Jin et al., 2020). However, to our best knowledge, no existing works have applied TTN to QML. Besides, since the tensor network-based machine learning model like TTN is closely related to quantum machine learning in terms of their model structures (Liu & Wang, 2018; Gao et al., 2017), the QTN-VQC model can be directly regarded as the classical simulation of the corresponding quantum machine learning. In addition to a classical dense layer, more complicated architectures like AlexNet (Lloyd et al., 2020) could be used for dimension reduction, and we also compare the performance between TTN and AlexNet-based models.
sections.heading: 3 NOTATIONS
sections.text: We denote RI as a I-dimensional real coordinate space, and RI1×I2×···IK refers to a space of Korder tensors. The symbol W ∈ RI1×I2×···×IK represents a K-order multi-dimensional tensor in RI1×I2×···IK , and the symbols v ∈ RI and W ∈ RI×J represent a vector and a matrix, respectively. For the notations of quantum computing, ∀ v ∈ RI , the symbol |v〉 denotes a quantum state associated with a 2I -dimensional vector in a Hilbert space. Particularly, |0〉 = [1 0]T and |1〉 = [0 1]T . The quantum gate RY (θ) means a Pauli-Y gate with a unitary operator as defined in Eq. (1), which implies a qubit rotates the Bloch sphere along the Y-axis by a given angle θ.
  RY (θ) =
  [ cos θ2 −i sin θ 2
  −i sin θ2 cos θ 2
  ] (1)
  Moreover, the operator ⊗ is a tensor product. Given the vectors vi ∈ RI , the tensor product of I vectors is defined as ⊗Ii=1vi, which is a 2I -dimensional vector and can provide a compact represen-
  tation for v1 ⊗ v2 ⊗ · · · ⊗ vI . Similarly, the symbol |0〉⊗S means a tensor product of S quantum states of |0〉. Furthermore, for a scalar v, the quantum state |v〉 can be written as:
  |v〉 = cos v|0〉+ sin v|1〉 = [ cos v sin v ] . (2)
sections.heading: 4 QTN-VQC: OUR PROPOSED END-TO-END LEARNING FRAMEWORK
sections.text: This section introduces our proposed end-to-end learning framework, namely QTN-VQC in this work. As shown in Figure 3, the QTN model includes two components (a) TTN and (b) TPE, which will be separately introduced in Section 4.1 and Section 4.2. Moreover, Figure 4 illustrates the framework of VQC and Section 4.3 is devoted to discussing the details of VQC.
sections.heading: 4.1 TENSOR TRAIN NETWORK FOR DIMENSION REDUCTION
sections.text: We leverage TTN (Novikov et al., 2015) for the dimension reduction of input features. TTN relies on the TT decomposition (Oseledets, 2011) and has been commonly employed in machine learning tasks like speech processing (Qi et al., 2020b) and computer vision (Yang et al., 2017). The TT decomposition assumes that given a set of TT-ranks {r0, r1, ..., rK}, a K-order tensor W ∈ RI1×I2×···×IK is factorized into the multiplication of 3-order tensors Xk ∈ Rrk−1×Ik×rk . In more detail, given a set of indices {i1, i2, ..., iK}, X (i1, i2, ..., iK) is decomposed as:
  X (i1, i2, ..., iK) = K∏ k=1 Xk(ik), (3)
  where ∀ik ∈ [Ik], Xk(ik) ∈ Rrk−1×rk . Since r0 = r1 = 1, the term ∏K k=1 Xk(ik) is a scalar value.
  TTN employs the TT decomposition in a dense layer and is explicitly demonstrated in Figure 3 (a). In more detail, for an input tensor X ∈ RI1×I2×···×IK and an output tensor Y ∈ RJ1×J2×···×JK , we achieve
  Y(j1, j2, ..., jK) = I1∑ i1=1 I2∑ i2=1 · · · IK∑ iK=1 W ((i1, j1), (i2, j2), ..., (iK , jK))X (i1, i2, ..., iK)
  = I1∑ i1=1 I2∑ i2=1 · · · IK∑ iK=1 ( K∏ k=1 Wk(ik, jk) ) · K∏ k=1 Xk(ik)
  = K∏ k=1 ( Ik∑ ik=1 Wk(ik, jk)Xk(ik) )
  = K∏ k=1 Yk(jk),
  (4)
  where Xk(ik) ∈ Rrk−1×rk , and Yk(jk) ∈ Rrk−1×rk which results in a scalar ∏K k=1 Yk(jk) because of the ranks r0 = r1 = 1; W((i1, j1), (i2, j2), ..., (iK , jK)) is closely associated with W(m1,m2, ...,mK) as defined in Eq. (3), if each index mk = ik× jk is set. The multi-dimensional tensorW is decomposed into the multiplication of 4-order tensorsWk ∈ Rrk−1×Ik×Jk×rk . A nonlinear activation function, e.g., Sigmoid, Tanh, and ReLU, is imposed upon the tensor Y . Compared with a dense layer with ∏K k=1 IkJk parameters, a TTN owns as few as ∑K k=1 rkrk−1IkJk trainable parameters.
  When a TTN is utilized for the dimension reduction, the high-dimensional input vector x ∈ RI is first reshaped into a tensor X ∈ RI1×I2×···×IK , and then we can represent X as a TT format that goes through TTN. The outputs of TTN can be converted back to a tensor Y ∈ RJ1×J2×···×JK , which is further reshaped to a lower dimensional vector y ∈ RJ . Here, we define
  ∏K k=1 Ik = I and∏K
  k=1 Jk = J . Moreover, the computational complexities of TTN and the related dense layer are in the same scale, which is discussed in (Yang et al., 2017).
  Eq. (4) suggests that TTN is a multi-dimensional extension of a dense layer, where the trainable weight matrix of a dense layer is changed to the learnable core tensors. Additionally, many empirical studies demonstrate that a TTN is capable of maintaining the baseline results of the dense layer (Qi et al., 2020b; Yang et al., 2017; Novikov et al., 2015; Qi et al., 2020a). More significantly, since TTN can be flexibly mapped into a quantum circuit, the quantumness inherent in TTN brings great advantages over other architectures like the dense layer. In other words, although TTN is treated classically, it is possible to substitute equivalent quantum circuits for TTN when more qubits become available (Du et al., 2020), which implies that QTN-VQC stands for a genuine end-to-end QNN learning architecture on a quantum computer.
  Furthermore, since the gradient exploding and diminishing problems are serious issues in the TTN training. To avoid those training problems, we only consider 3-order core tensors and small TT-ranks to configure a simple TTN in our experimental simulations. Our theoretical analysis of QTN-VQC based on Theorem 3 in Section 5 suggests that the representation power is not related to TT-ranks and the tensor order K, thus small TT-ranks and the tensor order K are preferred. In particular, a lower K can significantly reduce the computational cost and speed up the convergence rate.
sections.heading: 4.2 TENSOR PRODUCT ENCODING
sections.text: In this subsection, we first introduce Theorem 1, and then we derive our TPE associated with the circuits in Figure 3 (b). Theorem 1. Given the classical vector x = [x1, x2, ..., xI ]T ∈ RI , a TPE as shown in Figure 3 (b) can result in a quantum state |x〉 with the following complete vector representation as:(
  ⊗Ii=1RY (2xi) ) |0〉⊗I = [ cosx1 sinx1 ] ⊗ [ cosx2 sinx2 ] ⊗ · · · ⊗ [ cosxI sinxI ] = |x〉. (5)
  Proof. Since each element xi in the vector x can be written as |xi〉 = cosxi|0〉 + sinxi|1〉, the quantum state |x〉 can be written as:
  |x〉 = [ cosx1 sinx1 ] ⊗ [ cosx2 sinx2 ] ⊗ · · · ⊗ [ cosxI sinxI ] . (6)
  When the vector x goes through the quantum tensor network, which implies the following as: RY (2xi)|0〉 = cosxi|0〉+ sinxi|1〉 = |xi〉. (7)
  The preceding equation, in turn, implies that Eq. (5).
  Theorem 1 builds a connection between the vector x and the quantum state |x〉, and the resulting |x〉 is taken as the quantum embedding as the inputs to VQC. Since ⊗Ii=1RY (2xi) is a reversely unitary linear operator, there is no information loss incurred during the stage of quantum encoding. Furthermore, if the input is multiplied with a constant π2 , we obtain the following term as:(
  ⊗Ii=1RY (πxi) ) |0〉⊗I = [ cos(πx1) sin(πx1) ] ⊗ [ cos(πx2) sin(πx2) ] ⊗ · · · ⊗ [ cos(πxI) sin(πxI) ] , (8)
  which corresponds to Figure 3 (b).
  |0⟩
  |0⟩
  |0⟩
  |0⟩
  ⨁· · · ·⨁⨁ ⨁
  RX(α1) RY(β1) RZ(γ1)
  RX(α2) RY(β2) RZ(γ2)
  RX(α3) RY(β3) RZ(γ3)
  RX(α4) RY(β4) RZ(γ4)
  Z1
  Z2
  Z4
  Z3
  (a) Variational quantum circuit: the dashed square indicates repeated model with CNOT gates for entangling quantum states, and , , represent Pauli-X, Y, Z gates with free parameters RX(α) RY(β) RZ(γ) α, β, γ
  RY(β) = cos β2 −sin β 2
  sin β2 cos β 2
  RX(α) = [ cos α2 −i sin α 2
  −i sin α2 cos α 2
  ] RZ(γ) = exp(−i γ2 ) 0
  0 exp(i γ2 )
  Pauli-XPauli-Y Pauli-Z
  = 1 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0⨁ · CNOT gate
  (b) Matrix representation for the Quantum gates applied in the VQC
  Figure 4: A framework of variational quantum circuit.
sections.heading: 4.3 THE FRAMEWORK OF VARIATIONAL QUANTUM CIRCUIT
sections.text: The framework of VQC is shown in Figure 4 (a), where 4 qubit wires are taken into account, and the CNOT gates aim at mutually entangling the channels such that |x1〉, |x2〉, |x3〉 and |x4〉 lie in the same entanglement state. The Pauli-X, Y, Z gates RX(·), RY (·) and RZ(·) with learnable parameters (α1, β1, γ1), (α2, β2, γ2), (α3, β3, γ3), (α4, β4, γ4) are built to set up the learnable part. Being similar to the unitary operators of RY (α), RX(β) and RZ(γ), which are defined in Figure 4 (b), are separately associated with the rotations along X-axis and Z-axis by the given angles of β and γ. Besides, the quantum circuits in the dash square can be repeatedly copied to compose a deeper architecture. The outputs of VQC are connected to the measurement which projects the quantum states into a certain quantum basis that becomes a classical scalar zi.
  As for the end-to-end training paradigm for QTN-VQC, the learnable parameters come from the VQC and TTN models, and they should be updated by applying the back-propagation algorithm based on the Adam optimizer. Given D qubits and H depths, there are totally 3DH trainable parameters for VQC. Consequently, there are ∑K k=1 rk−1rkIkJk + 3DH parameters for QTN-
  VQC. On the other hand, the Dense-VQC model possesses more model parameters than QTN-VQC ( ∏K k=1 rk−1rkIkJk + 3DH vs. ∑K k=1 rk−1rkIkJk + 3DH).
sections.heading: 5 CHARACTERIZING REPRESENTATION POWER OF QTN-VQC
sections.text: This section focuses on analyzing the representation power of QTN-VQC. As shown in Figure 5, given d qubits and a target quantum state |z〉 = ⊗Dd=1|zd〉, sinceHθ is known as a linear operator and Tx is defined as a definite mapping from input x to the unitary matrix Ux, the representation power of QTN-VQC is determined by how TTN can approximate the classical vector T −1x (H−1θ |z〉). To understand the expressiveness of TTN, we first start with the discussion on the expressive capability of Dense-VQC (a dense layer is taken for dimension reduction) and then generalize it to QTN-VQC. Based on the universal approximation theorem (Cybenko, 1989; Barron, 1994) for a feed-forward neural network, we derive the following theorem as: Theorem 2. Given a target vector T −1x (H−1θ |z〉), there exists a feed-forward neural network fdense with a dense layer connecting to D qubits, then
  || T −1x (H−1θ |z〉)− fdense(y) ||1 ≤ C√ D , (9)
  where the activation function tanh(·) is imposed upon the dense layer, and C is a constant associated with the target vector T −1x (H−1θ |z〉).
  Since TTN is a compact TT representation of a dense layer, by modifying Theorem 2 for TTN, we can also derive the upper bound on the approximation error as follows:
  Theorem 3. Given a target vector T −1x (H−1θ |z〉), there exists a TTN, denoted as fTTN , with a TT layer connecting to D qubits, then
  || T −1x (H−1θ |z〉)− fTTN (y) ||1 ≤ K∏ k=1 C√ Dk , (10)
  where ∏K k=1Dk = D, the Sigmoid activation function is imposed upon the TTN model, K denotes the multi-dimensional order, C is a constant associated with the target vector T −1x (H−1θ |z〉).
  Comparing the two upper bounds, it is observed that TTN can attain an identical upper bound as the dense layer on the approximation error because ∏K k=1Dk = D. That implies that TTN can at least maintain the representation power of a dense layer. Besides, the number of qubits D is a key factor determining the upper bound on the approximation error. However, D is a small fixed number on a NISQ device, and a larger number of qubits D is expected to further improve the representation power of QTN-VQC. However, the computational costs of classical simulation may grow exponentially with the increasing number of qubits, and a small number of qubits have to be considered in practice.
sections.heading: 6 EXPERIMENTS AND RESULTS
sections.heading: 6.1 EXPERIMENTAL SETUPS
sections.text: We assess our QTN-VQC based end-to-end learning system on the standard MNIST. MNIST is a dataset for the task of 10 digit classification, where there are 50000 and 10000 28 × 28 image data assigned for training and testing, respectively. The full MNIST dataset is challenging for quantum machine learning algorithms, and many works only consider 2-digit classification on the MNIST task (Wang et al., 2021; Chen et al., 2020a). Moreover, the image data are separately reshaped into 784 dimensional input vectors. Dense-VQC and PCA-VQC are taken as our experimental baselines to compare with our QTN-VQC model. Dense-VQC denotes that a dense layer is used for dimension reduction, and PCA-VQC refers to using principal component analysis (PCA) to extract low-dimensional features before training the VQC parameters.
  As for the experiments of QTN-VQC, the image data are reshaped into 3-order 7× 16× 7 tensors. We set small TT-ranks as {1, 2, 2, 1} to reduce the computational cost of TTN. the image data are represented as the TT format according to Eq. (3) before going through the TTN model. Since 8 qubits are used for the quantum encoding, the output of TTN needs to configure the tensor format as 2 × 2 × 2, which results in 8 dimensional output vectors. Besides, the model parameters of QTN-VQC are randomly initialized based on the Gaussian distribution, and the back-propagation algorithm is applied to train the models. The Sigmoid function is utilized for the hidden layers of TTN.
  To be consistent with QTN-VQC, the weight of the dense layer for Dense-VQC is configured as the shape of 784×8. Although Dense-VQC is a hybrid classical-quantum model, the training process of Dense-VQC can also be set as an end-to-end pipeline and the weights of the dense layer are updated during the training stage. The Sigmoid function is used for the dense layer. On the other hand, PCA
  is employed to reduce the feature dimension to 8, and the resulting low-dimensional features are further encoded into quantum states. Consequently, PCA-VQC admits the VQC parameters solely to be updated during the training stage. A standard AlexNet (Iandola et al., 2016) is employed to constitute an AlexNet-VQC to compare the performance.
  Moreover, 6 VQC layers are constructed to form a deep model, and the outputs of the VQC model are connected to 10 classes with a non-trainable matrix. The back-propagation algorithm based on the Adam optimizer with a learning rate of 0.001 is employed for the model training. The loss of cross-entropy (CE) is utilized as the objective function during the training stage, and it is also taken as the metric to evaluate the model performance. We leverage the tools of Pennylane (Bergholm et al., 2018) and PyTorch (Paszke et al., 2019) to simulate the model performance. In particular, we separately simulate the model performance with noiseless quantum circuits and noisy quantum circuits corrupted by quantum noises from IBM quantum machines.
sections.heading: 6.2 EXPERIMENTAL RESULTS OF NOISELESS QUANTUM CIRCUIT
sections.text: Table 1 shows the final results of the models on the test dataset. QTN-VQC owns much fewer model parameters than Dense-VQC (328 vs. 6416) and attains even higher classification accuracy than Dense-VQC (91.43% vs. 88.54%) and lower loss values than Dense-VQC (0.3090 vs. 0.4132). However, PCA-VQC with 144 trainable VQC parameters attains the worst performance by all metrics, which implies that a trainable quantum embedding is of significance to boost experimental performance. Although our empirical results cannot reach the state-of-the-art classification performance of classical ML algorithms, our empirical results demonstrate the advantages of QTN-VQC over the PCA-VQC and Dense-VQC counterparts. With the development of more powerful quantum devices supporting more qubits, the representation power of QTN-VQC can be improved and better experimental results could be attained. Moreover, AlexNet-VQC achieves better results than QTN-VQC (92.81%vs.91.43%), but it involves more model parameters than QTN-VQC.
sections.heading: 6.3 EXPERIMENTAL RESULTS OF NOISY QUANTUM CIRCUIT
sections.text: To empirically validate the effectiveness of our proposed VQC algorithm, we proceed with the simulation of the practical experiments with noisy quantum circuits. More specifically, we follow an established noisy circuit experiment with the NISQ device suggested by (Chen et al., 2020b). One major advantage of the setups is to observe the robustness and preserve the quantum advantages of a deployed VQC with physical settings being close to quantum processing unit (QPU) experiments without an executive queuing time. As for the detailed setup, we first use an IBM Q 20-qubit machine to collect channel noise in the real scenario for a deployed VQC and upload the machine noise into our Pennylane-Qiskit simulator (denoted as Accq20. We provide a depolarizing noisy circuit simulation (denoted as Accdepo) based on a depolarizing channel attained from (Nielsen & Chuang, 2010) with a noise level of 0.1. As shown in Table 2, the quantum noise brings about the performance degradation of all models, but our proposed QTN-VQC consistently outperforms PCA-VQC and Dense-VQC in the condition of noisy quantum circuits. In particular, QTN-VQC can even outperform the AlexNet-VQC counterpart in noisy circuit conditions.
sections.heading: 6.4 FURTHER DISCUSSIONS
sections.text: The above experimental results show the advantages of QTN-VQC over Dense-VQC and PCAVQC in the scenarios with noiseless and noisy quantum circuits. Next, we will further discuss the representation power of QTN-VQC based on two factors: (1) the activation function used in TTN; (2) the number of qubits.
sections.heading: 6.4.1 THE ACTIVATION FUNCTION USED IN TTN
sections.text: Table 3 compares the results of QTN-VQC based on different activation functions. Our simulation on noiseless quantum circuits shows that the non-linear activation functions can bring more performance gain than a linear one, but the Sigmoid function attains a better performance than the Tanh and ReLU counterparts in our experiments. Our experiments also correspond to the universal approximation theory for QTN-VQC in Theorem 3.
sections.heading: 6.4.2 THE NUMBER OF QUBITS
sections.text: Finally, we investigate the effects of the number of qubits on the performance of QTN-VQC by increasing the qubits from 8 to 12 and 16. Accordingly, the output of TTN is configured as a tensor format of 2 × 3 × 2, and the model size is increased from 328 to 464 and 600, respectively. Our experiments show that the baseline performance of QTN-VQC can be further improved by increasing the number of qubits, which implies that more qubits are likely to possess higher accuracy.
sections.heading: 7 CONCLUSIONS
sections.text: This work proposes a genuine end-to-end learning framework for quantum neural networks based on QTN-VQC. QTN consists of a TTN for dimension reduction and a TPE framework for generating quantum embedding. The TTN model is a compact representation of a dense layer to classically simulate quantum machine learning algorithms. Our theorem on the representation of QTN-VQC shows that the number of qubits is inversely related to the approximation error of QTN-VQC and the non-linear activation plays an important role. Our experiments compare our proposed QTN-VQC with Res-VQC, Dense-VQC, and PCA-VQC. Our simulated results demonstrate that QTN-VQC obtains better experimental performance than Dense-VQC and PCA-VQC with both noiseless and noisy quantum circuits, and it achieves marginally worse performance than AlexNet-VQC. Besides, our results justify our theorem on the representation power of QTN-VQC.
sections.heading: A APPENDIX
sections.text: The section of appendix includes the proofs for Theorem 2 and Theorem 3.
  A.1 PROOF FOR THEOREM 2
  Proof. Theorem 2 is derived from the modification of the universal approximation theory proposed by Barron (1994); Cybenko (1989). The universal approximation theory is shown in Lemma 1, which suggests that a feed-forward neural network with d neurons can approximate any continuous function with arbitrarily small .
  Lemma 1. Given a continuous target function f̂ : RI → R, we can employ a 2-layer neural network with a non-linear activation f : RI → R, such that
  |f̂ − f | ≤ 1√ D Cf̂ , (11)
  where J denotes the number of neurons, and Cf̂ is a constant associated with f̂ . In particular, for r ≥ 1, Cf satisfies the following condition as:
  ||f̂ ||∞ + ∑
  k,1≤ k(k−1)2 ≤r
  ||Dkf̂ ||∞ ≤ Cf̂ , (12)
  where Dkf̂ = [ ∇f̂ ,∇2f̂ , ...,∇kf̂ ]T .
  To associate Lemma 1 with our Theorem 2, the target function is replaced with the target vector H−1X H −1 θ |z〉, then there is a neural network with a dense layer connected to D qubits such that
  ||H−1X H −1 θ |z〉 − fdense(y)||1 ≤ 1√ D C, (13)
  where C is related to the target vector H−1X H −1 θ |z〉.
  A.2 PROOF FOR THEOREM 3
  Proof. Assume that X = fTTN (y), X̂ = H−1X H −1 θ |z〉 and the TT decomposition of target vector is {X̂1, X̂2, ..., X̂K}, then we obtain
  ||H−1X H −1 θ |z〉 − fTTN (y)||1 = ||X̂ − X ||1 ≤ K∏ k=1 ||X̂k −Xk||1 (14)
  On the other hand, we denote vec(Yk) and vec(Xk) as the vectorization of the tensors Yk and Xk, respectively. We also define ∏K k=1 Ik = I , Wk ∈ RDk×Ik×rk−1×rk as the TTN parameters, and also define Wk ∈ RIk×rk−1rkDk as the matricization of Wk. Moreover, σ refers to a non-linear activation function.
  Since vec(Xk) = σ(WT vec(Yk)) that corresponds to a dense layer, we can obtain that
  ||X̂k −Xk||1 ≤ 1√ Dk C. (15)
  In sum, we can further obtain
  ||H−1X H −1 θ |z〉 − fTTN (y)||1 ≤ K∏ k=1 ||X̂k −Xk||1 ≤ K∏ k=1 1√ Dk C, (16)
  where ∏K k=1Dk = D.
sections.heading: B APPENDIX
sections.text: This section includes additional experimental simulations. First, we assess the settings of TT-ranks, and then we compare the convergence rates of QTN-VQC and Dense-VQC in the experiments.
  B.1 EXPERIMENTS ON TT-RANKS FOR QTN-VQC
  Table 5 corresponds to the experiments of QTN-VQC with 8 qubits and the Sigmoid function. The empirical results suggest that the larger TT-ranks cannot result in better results than the smaller ones. The main reason is that the TT-ranks can correspond to a manifold, and there may potentially exist an optimal manifold with smaller TT-ranks that corresponds to the best performance.
  B.2 A COMPARISON OF CONVERGENCE RATES
  Next, we analyze the computational complexity for TTN for QTN-VQC. In more detail, given the TT-ranks {r1, r2, ..., rK}, a multi-dimensional tensorW is factorized into several K-order tensors Wk ∈ Rrk−1×Ik×Jk×rk , the computational complexity of the feed-forward process is in the scale of O(Kmaxk Ikmaxk Jk(maxk rk)K). In contrast, the computational overhead for a dense layer is in the scale of O( ∏ k Ik ∏ k Jk). It means that smaller TT-ranks can reduce the computational cost for QTN-VQC, which explains that smaller TT-ranks {1, 2, 2, 1} is configured in our experiments of QTN-VQC.
  Empirically, we compare the convergence rates of different models on the test data in our experiments. In our experimental settings with the Tanh activation function and 8 qubits, the QTN-VQC model consistently attains a faster convergence rate than the Dense-VQC and PCA-VQC counterparts. Moreover, Table 6 compares the absolute running time of QTN-VQC with Dense-VQC and AlexNet-VQC. Since our experiments are conducted on the same GPUs and CPUs, the training time of all models can be comparable. Our evaluation shows that QTN-VQC is marginally slower than Dense-VQC, but it is much faster than AlexNet-VQC.
sections.heading: C EXPERIMENTS OF LABELED FACES IN THE WILD (LFW)
sections.text: C.1 EXPERIMENTAL SETUPS
  The LFW is a dataset for the task of unconstrained face recognition, which is composed of 13000 images with the shape of [154, 154, 3]. The shape of We randomly split all the datasets into 11000 training data, 2000 test data. 16 qubits are used for VQC, and the shape of the input tensor is set as 22× 147× 22. The other settings are kept the same as the configurations for the MNIST task.
  C.2 EXPERIMENTAL RESULTS
  Table 6 presents the simulation results under the noiseless quantum circuit condition, while Table 7 demonstrates the empirical results in the setting of noisy quantum circuits. The QTN-VQC outperforms the Dense-VQC counterpart (92.15 vs. 91.27), and it owns much fewer model parameters (2816 vs. 1.1× 106). Although The experimental results on the LFW dataset also highlight the advantages of QTN-VQC in terms of fewer model parameters and better empirical performance.