Hi,
All the losses seem to share the same computation graph. So to be able to use that graph multiple times when you do multiple backward calls, you have to keep it around with retain_graph=True. I don’t see any way around this but reduce the memory usage by reducing batchsize.