Pytorch690×626 4.17 KB

Sorry for my terrible drawing. But according to the above figure, I want to use 2 separate losses to update different modules in the entire network. The data flow is illustrated with arrows in the above figure, I hope to define a loss1 to only update “trainable 1”, and another term loss2 to only update “trainable 2”.
I apologize because I can’t paste our internal code here, so I have to briefly mention what I did:
I defined two optimizers: optimizer_1 (with parameters in “trainable 1” as trainable variables) and optimizer_2 (with parameters in “trainable 2” as trainable variables).
For each iteration of my training loop, I did something like this:
optimizer_1.step()
optimizer_2.step()
loss1.backward()
loss2.backward()

And crashed with the following error message
# [RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time]
I am not sure if I should handle these two losses in this way. Could someone tell me what  might be potentially wrong in my implementation, and how to fix it?
Thanks.