matmul is implemented inefficiently in PyTorch when it behaves like a batched matrix multiply and requires broadcasting. Please file a bug report.
The first case (i1.matmul(W)) is saved by a special-case in PyTorch that treats a (B x M x K) @ (K x N) as a matrix multiply (B*M x K) @ (K x N)
The second case expands the 768x768 tensor into a 5000x768x768 tensor which is huge.
matmul() isn’t associative because floating point arithmetic isn’t associative 15. This is generally the case and not specific to PyTorch.