I think this is expected. There is some overhead in launching CUDA code and you are launching many kernels, and in your case the matrices are very small so there is no benefit in using GPUs over CPUs.
Note that you can use torch.bmm to perform batched matrix multiplication, and it will be orders of magnitude faster than a for loop.