I had similar experience with libtorch. Looking at your code, you are not using any libtorch specific functionality that binds to using tensors only. So you should be able to easily swap tensor with a vector.
Just converting the mult1 from a tensor into a vector, would greatly boost your performance.