Your benchmarking is wrong:

You’re including the time it takes to copy the data into a tensor
You’re including the time it takes to copy the tensor to the GPU
Many CUDA calls are asynchronous and you’re not taking that into account
You’re using int16, which isn’t a common use case. I’m not sure how that affects perf, but it’s not a common target in PyTorch.

EDIT: It took me a while to realize that “multiplying matrices” meant element-wise multiplication not matrix multiplication.