I;m building recursive neural nets, so I’ll have a variable list of criterions. I want to sum them up and backpropagate error.
All the errors are single float values and of same scale. So the total loss is the sum of individual losses
To understand this, I crystallized this problem. It’s for another classification project.
I wrote this code and it works
def loss_calc(data,targets):
  data = Variable(torch.FloatTensor(data)).cuda()
  targets = Variable(torch.LongTensor(targets)).cuda()
  output= model(data)
  final = output[-1,:,:]
  loss = criterion(final,targets)
  return loss

Now I want to know how I can make a list of criterions and add them for backpropagation.
I tried these, but they don;t work
def loss_calc(data,targets):
  data = Variable(torch.FloatTensor(data)).cuda()
  targets = Variable(torch.LongTensor(targets)).cuda()
  output= model(data)
  final = output[-1,:,:]
  loss = 0.0
  for b in range(batch_size):
    loss += criterion(final[b], targets[b])
  return loss

I doesn;t return errors, but the network simply won;t train.
This alternative results an error
def loss_calc(data,targets):
  data = Variable(torch.FloatTensor(data)).cuda()
  targets = Variable(torch.LongTensor(targets)).cuda()   
  output= model(data)
  final = output[-1,:,:]
  loss = []
  for b in range(batch_size):
    loss.append(criterion(final[b], targets[b]))
  loss = torch.sum(loss)
  return loss

Note, this is a dummy example. If I understand how to fix this, I can apply that to the recursive neural nets.
For the final project, the sequence of losses have arbitrary length. For example, sometimes it’s adding 4 losses, other times 6. So it’s not an option to pre-allocate a Tensor and save the losses to the Tensor