Hi all,
Say I calculate 4 losses for each batch as shown below:
losses_list = []
some_losses_fns = [loss_fn1,loss_fn2,loss_fn3,loss_fn4]
for loss_fn in some_losses_fns:
  # assume loss is computed 
  loss = loss_fn(preds, true_labels)
  losses_list.append(loss.item())
average_loss = torch.cat(losses_list).mean()
average_loss.requires_grad_()

Are the gradients of the losses going to be backpropagated properly if when I call .item() on them and then call requires_grad_() on the average loss?
I am trying to reduce the memory footprint of my model and I found this suggestion online but I wanted to double check whether it is legit. Any other suggestions? Maybe del the loss variable after it is appended to the losses_list?
Thanks in advance!