Yes, you should be able to do that. It looks one of your sizes doesn’t match up, but it’s hard to tell where from your snippet. Can you post a link to a full working example?
Here’s a simple snippet showing multiple calls to backards:
import torch
a = torch.autograd.Variable(torch.randn(5, 5), requires_grad=True)
b = torch.autograd.Variable(torch.randn(5, 5), requires_grad=True)
c = a @ b

for i in range(5):
  cost = c[i,:].sum()
  cost.backward(retain_variables=True)