  States of quantum many-body systems are defined in a high-dimensional Hilbert
space, where rich and complex interactions among subsystems can be modelled. In
machine learning, complex multiple multilinear correlations may also exist
within input features. In this paper, we present a quantum-inspired multilinear
model, named Residual Tensor Train (ResTT), to capture the multiple multilinear
correlations of features, from low to high orders, within a single model. ResTT
is able to build a robust decision boundary in a high-dimensional space for
solving fitting and classification tasks. In particular, we prove that the
fully-connected layer and the Volterra series can be taken as special cases of
ResTT. Furthermore, we derive the rule for weight initialization that
stabilizes the training of ResTT based on a mean-field analysis. We prove that
such a rule is much more relaxed than that of TT, which means ResTT can easily
address the vanishing and exploding gradient problem that exists in the
existing TT models. Numerical experiments demonstrate that ResTT outperforms
the state-of-the-art tensor network and benchmark deep learning models on MNIST
and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than
other statistical methods on two practical examples with limited data which are
known to have complex feature interactions.

Machine Learning