Hello,
I’m performing a matrix multiplication using matmul function:
hidden_size = 8
batch_size  = 5

W = Var(hidden_size,hidden_size)
emb = torch.tensor(torch.randn(batch_size,12,hidden_size))

res =  emb.matmul(W).bmm(emb.transpose(1, 2))

The first matmul fuction just broadcast the operation within the batch dimensions and the result is as expected.
Another way of accomplishing this is using bmm:
Wb = W.expand(batch_size,  -1,-1)
resb = emb.bmm(Wb).bmm(emb.transpose(1, 2))

both results are equal
torch.norm(res2-res)
but while training my model, the loss has different training values using the first or the second method.
I’m guessing that the two methods generate different computational graphs.
Is it?
If so, what is the correct way of doing this?
Thank you in advance,
Ricardo