@Caruso answer above is good for 1.
for 2: I’m afraid there is no such thing. But there are functions to do all the matrix ops that you want in general so hopefully such a guide is not necessary. In general, I use torch.matmul() that performs generic batch matrix multiplication. And add extra dimensions where needed.
Note that sometimes, it is more efficient to do the product reduction by hand and you can do an element-wise product and a sum(dim=[-1, -2]) for example if you need to reduce two dimensions at once.
Finally, if you are interested, we also have a einsum() function that allows you to perform arbitrary reductions specified with Einstein notation.
For 3:  PyTorch / CUDA efficiency rules are the same as regular CUDA efficiency rules (that you can find online). The gist is: do only large ops