miturian:

it seems that for now, I should just stay away from einsum?


You do whatever keeps PyTorch most useful for you. But beware: The different implementations of matmul can and will have numerical precision differences, too.
The mathematician in me is convinced that we’ll eventually have more robust procedures, but clearly we aren’t there yet.
Best regards
Thomas